| I'm afraid you bitterly underappreciate the amount of unanticipated connections of everything with everything else in a complex system. An industrial steam engine will never explode like a bomb, unless explicitly designed to do so. An agricultural insecticide will never accumulate in human bodies, unless explicitly designed to do so. A speculative execution unit will never reveal data from a privileged process, unless explicitly designed to do so. A toy quadcopter will never be able to carry a lethal weapon, unless explicitly designed to do so, An LLM will never tell outright lies, or engage in racial prejudices, unless explicitly designed to do so. Oh wait. Even when you explicitly try to make some states impossible in a complex system, often a parasitic connection or a benign-looking failure mode re-enables the thing you tried hard to disable. If you just ignore it because "it's impossible anyway", without active suppression, the chances of a nasty surprise become quite high. If the blind watchmaker of the biological evolution produces self-awareness here and there, the probability that you may encounter some variety of it while stomping all over the territory of intelligent machines that are fed the sum total of human knowledge should be close to 1. |