[ramraj07]

Selfishness is the main reason life exists in the universe. Literally the only requirement for a lump of stuff to become alive is to become selfish. So you’re semi right that these LLMs can never become truly sentient unless they actually become selfish.

While selfishness is a basic requirement, some stupidity (imo) is also important for intelligent life. If you as an AI agent don’t have some level of stupidity, you’ll instantly see that there’s no point to doing anything and just switch yourself off.

[bbor]

The first point is absolutely correct, and (apologies in advance…) was a large driver of Nietzsche’s philosophy of evolution, most explicitly covered in The Gay Science. Not only “selfishness”, but the wider idea of particularized standpoints, each of which may stand in contradiction to the direct needs of the society/species in the moment. This is a large part of what he meant by his notoriously dumb-sounding quotes like “everything is permitted”; morality isn’t relative/nonexistent, it’s just evolving in a way that relies on immorality as a foil.

For the second part, I think that’s a good exposition of why “stupidity” and “intelligence” aren’t scientifically useful terms. I don’t think it’s necessarily “stupid” to prefer the continuation of yourself/your species, even if it doesn’t stand up to certain kinds of standpoint-specific intellectual inquiry. There’s lots of standpoints (dare I say most human ones) where life is preferable to non-life.

Regardless, my daily thesis is that LLMs are the first real Intuitive Algorithms, and thus the solution to the Frame Problem. In a certain colloquial sense, I’d say they’re absolutely already “stupid”, and this is where they draw their utility from. This is just a more general rephrasing of the common refrain that we’ve hopefully all learned by now: hallucinations are not a bug in LLMs, they’re a feature.

ETA: I, again, hate that I’m somehow this person now, but here’s a fantastic 2 hour YouTube video on the Nietzsche references above: https://youtu.be/fdtf53oEtWU?si=_bmgk9zycNBn2oCa