[captainclam]

Haha, I've had the same thoughts, that of course computers/AI/droids of that conversational capacity were conscious. You'd be a brute not to think that!

And all of a sudden, LLMs absolutely have the command of natural language that once seemed such an obvious indicator of sentience, and now I find myself one of those bigots who don't believe in robot rights!

I'm being silly, but I do think there are implications here with respect to the future debate on AI sentience. I guess I once thought there would be this threshold where the reality of an AI's inner experience became blatantly obvious, but I see now that this is going to be a profoundly thorny problem.

Who knows, maybe in several decades we'll have a consciousness-o-meter that demonstrates that LLMs have had some degree of awareness all along.

[ANewFormation]

The fundamental problem is that we keep setting 'reasonable' benchmarks, implicitly because we want to make them seem reachable, but then when we inevitably do we find it feels we [of course] haven't achieved what we really wanted to in the first place.

True artificial intelligence would have the ability to meaningfully and significantly improve itself, recursively - a characteristic of even the most primitive life on Earth.

And the implications of a computing system capable of such simply can't be overstated. With access to essentially all contemporary knowledge, flawless recall, and stupidly ridiculous amounts of energy to power itself (relative to e.g. a brain), it would create an explosion of knowledge in every single domain imaginable, and many we can't even yet fathom. And it would be doing this at an accelerating pace owing to self improvement.

Basically the very nature of existence and knowledge would change. Sentience or not would be irrelevant as we simply tried (and inevitably failed, miserably) to catch up.

But this is a bit further out of reach than 'can vaguely pass for a human in a casual non-adversarial q&a style chat.'

---

This was one of the rather many areas where Star Trek failed to really consider the implications of its concepts, probably because it would simply break the world building. The Borg, for instance, would be defacto Gods in no time, even without assimilation.

[captainclam]

> This was one of the rather many areas where Star Trek failed to really consider the implications of its concepts, probably because it would simply break the world building.

This is true of pretty much all scifi! It's funny seeing super-futuristic depictions of star-fighter pilots and combatants with firearms and its just...so crushingly evident that humans will not have supremacy in these arenas very shortly.

Frank Herbert must have anticipated this complication and side-stepped the whole issue by preemptively canonizing the Butlerian Jihad.

[readyplayernull]

> meaningfully and significantly improve itself

Exactly, mechanically by tools, environmentally by science, metaphysically by philosophy. An AI can only become an I by crossing a singularity of knowledge.

[kenjackson]

What seems far more likely, the way the world is going, is that we’ll use the consciousness-o-meter to determine some groups of humans are less conscious than others.

[koakuma-chan]

You don't need a consciousness to be aware.

[captainclam]

semantics shmemantics.