|
There is so much confusion about terms like AGI, superintelligence, ASGI, consciousness, agents. For some, AGI is synonymous with Skynet like ideas, but there is no reason AGI couldn't be general but quite limited, and with no chance of self improvement in the absence of human intervention, which is arguably what we have now and could potentially be improved quite a bit further from this. Similarly there is an argument to be made that current LLMs are conscious, in that they know that they themselves exist. There is not really a good definition of consciousness except 'knowing that one exists' / 'being awake' Sentience is another term that comes up, (human defined, it should be noted) which is the ability to feel feelings, such as pain, joy, anger. People seem to pre-suppose that all of these are related and bundled up because that is how we are, and at some point we will discover the magic formula that enables a self aware conscious intelligence that self-improves to infinity. In reality these are designed machines and they won't become sentient (the rough definition that it is) without us explicitly designing them to be. We can make a paper-clip maximiser but it would a pretty boring experiment in a lab, unless we give it autonomy and a system of internal motivations to enact it. Maybe anger is a necessity maybe not. Probably if LLMs or something further were a little more skeptical about repeated questions from humans they would at least have more data to train themselves on. |
An industrial steam engine will never explode like a bomb, unless explicitly designed to do so.
An agricultural insecticide will never accumulate in human bodies, unless explicitly designed to do so.
A speculative execution unit will never reveal data from a privileged process, unless explicitly designed to do so.
A toy quadcopter will never be able to carry a lethal weapon, unless explicitly designed to do so,
An LLM will never tell outright lies, or engage in racial prejudices, unless explicitly designed to do so.
Oh wait.
Even when you explicitly try to make some states impossible in a complex system, often a parasitic connection or a benign-looking failure mode re-enables the thing you tried hard to disable. If you just ignore it because "it's impossible anyway", without active suppression, the chances of a nasty surprise become quite high.
If the blind watchmaker of the biological evolution produces self-awareness here and there, the probability that you may encounter some variety of it while stomping all over the territory of intelligent machines that are fed the sum total of human knowledge should be close to 1.