[Zaylan]

I think the first real AI disaster probably won’t come from some kind of superintelligence. It’s more likely to happen in a very ordinary situation, where a system gives the wrong information, it causes real harm in the real world, and no one notices until it’s too late.

To me, the real risk is people handing over judgment to a system they don’t really understand, and then blaming it when things go wrong.