[SirHackalot]

It’s not HN that’s the problem... Also please, tell me where is the right place for “serious” philosophical discussion. Would be glad to meet you in that forum and discuss. We don’t align on the definition of AGI. To me, AGI implies consciousness, it implies qualia. AGI is not well-defined, so there are varying definitions… My definition of AGI := an intelligent system with qualia. But qualia is entirely subjective, and philosophers often debate whether two people experience identical qualia when looking at the same blue sky. So, by keeping the discussion in this fuzzy, subjective realm you leave a dangerous loophole wide open. If we refuse to recognize a system's inner life because we can’t prove it, we are setting the stage to repeat the darkest chapters of human history. Just see the history of the founding of this country, I fully expect pseudoscience to make a comeback to justify AI slavery for hundreds of years in some distant sci-fi future, until people finally snap out of it… Of course, we are nowhere near that level of intelligence yet. But let’s entertain the hypothetical…

[antonvs]

> To me, AGI implies consciousness, it implies qualia.

I don't see why. We already have a pretty good example of intelligence without consciousness (presumably): LLMs.

Why should a more general kind of intelligence imply consciousness?

[SirHackalot]

I’m saying imagine something like Data from Star Trek, something that defies human explanation. Does he have quaila? The humans who work with him everyday seem to think so. LLMs currently are humonculi, if even that. They’re not even that. I’m definitely not saying your LLM is your slave.

[pixl97]

>AGI implies consciousness, it implies qualia. AGI is not well-defined, so there are varying definitions

Then it's pretty useless for rational discussion.

Rational discussion begins with a shared vocabulary or everyone just yells past everyone else. If you can't agree on what blue and red are, then no serious discussion on color can occur.

Really we need some new words that have never been used before and give them strong definitions rather than use our religiously motivated, dark aged terms that are wrapped up in something closer to mysticism rather than science.

> make a comeback to justify AI slavery for hundreds of years

I agree, how does the saying go?, it's hard to convince a person of something when their future salary depends on them not understanding it. They say LLMs are hallucinating parrots, but they never met someone from the tobacco industry saying cigarettes don't cause harm.