Hacker News new | ask | show | jobs
by clueless 1324 days ago
We won't have AGI until we have an AI agent that feels the need to survive (i.e. tries not to "die"). It will be only then, meaning with a selfish sense of survival and the ability to better itself in its various mental models, that we'll have a chance at AGI and possibly a consciousness AGI at that.
1 comments
ly3xqhl8g9 1324 days ago
What is "feels" and what is "the need"? Does a virus, biochemical or digital, also "feels the need to survive"? And also, why would an AGI care about survival? Perhaps a higher intelligence than ours will contemplate how doomed the planet is, being evaporated by the sun in circa 5 billion years [1], and how doomed the universe is, being evaporated by proton decay(?) in 10^100 years [2], and once the AGI internalizes how hopeless everything is, they simply commit suicide.

It's also revealing of our times how our definition of intelligence is being able to do work: transform raw materials and free energy into tools and toys, handle the tools and toys in open environments. An AGI could perhaps want nothing to do with this strifling struggle.

[1] https://en.wikipedia.org/wiki/Timeline_of_the_far_future

[2] https://en.wikipedia.org/wiki/Graphical_timeline_from_Big_Ba...

link
ethanwillis 1324 days ago
This is extremely defeatist. If you're worried only about such things then of course the present and even short term future can't be enjoyed. Why does a goal in the far future matter more than a goal in the immediate future? (that isn't the heat death of the universe)
link
ly3xqhl8g9 1324 days ago
The point is that humans are not AGI and we don't know what would an AGI "feel" just as much as an ant has no way of knowing what a human "feels" when listening to an EDM remix of Chopin (excitement or sheer hatred?). But, yes, I do find funny the idea of the depressed AGI, thinking of Marvin the Paranoid Android [1].

[1] https://en.wikipedia.org/wiki/Marvin_the_Paranoid_Android

link