[Terribledactyl]

> We take full responsibility for not seeing this possibility ahead of time.

Not responsibility for the bot's actions, but responsibility for us not predicting what it would do. Subtle, but sets the tone for we're not responsible for our AI, it just did it.

But isn't part of the point of the AI to do things difficult to predict?

[theoh]

Just my opinion, but AI should be controllable and should probably start by, as the "Bayesian brain" theorists say, developing a robust and reliable model of its environment. The Bayesian brain aspect of this is the belief that what AI needs to do is to minimize its own prediction errors.

I take that to mean that an AI should learn to be safe and predictable on human terms before it is allowed to start to diverge from human expectations. Even that sounds a bit scary.