[N0b8ez]

>AGI would have to suffer to become general enough to warrant the use of the word general.

Jo Cameron doesn't have general intelligence? This seems absurd. Intelligence is orthogonal to phenomenology and affective states. People aren't worried about AGI because it might have a Cartesian theater, the worry is that it might be more competent than humans and put humans out of a job. The semantics of whether it "truly" has intelligence is irrelevant.

>Suffering is part of our intelligence and we’d be nowhere without it.

It might be part of our intelligence, but why would it need to be part of machine's intelligence? GPT4 is already beating humans at theory of mind tasks[1], and I doubt that it suffers. Our suffering is an evolutionary stroke of bad luck. It has nothing to do with intelligence itself, and it would have been better if we had evolved some other way that didn't need it.

[1] https://www.nature.com/articles/s41562-024-01882-z

[gizajob]

If LLMs are beating humans in theory of mind tasks, your theory of mind is incorrect.

I’d not heard of Jo Cameron but one outlier, for me, wouldn’t hyper-negate the whole idea out of existence. A huge amount of what I suffer with is not physical pain.

[N0b8ez]

> A huge amount of what I suffer with is not physical pain.

It's not just physical suffering that she's supposedly immune to. She's also immune to all psychological suffering as well. And she is married, has kids, and is perfectly healthy. If she's in fact the real deal (I don't know this myself, but I haven't seen anyone debunking her) then dismissing her as an outlier would be a remarkable form of complacency. If the effect can be understood and replicated, suffering will become fully as needless as it deserves to be, with its final eradication mourned probably as much as the death of smallpox.

> If LLMs are beating humans in theory of mind tasks, your theory of mind is incorrect.

Or, LLMs have better theory of mind than most humans do, which is the finding of that study. Is it metaphysically impossible? If your mental image of how LLM cognition works is the same type of expert system that Searle was writing about back in the day, it's refreshingly bizarre to read how LLMs appear to work:

https://www.astralcodexten.com/p/god-help-us-lets-try-to-und...

Where is the Chinese Room in this? I don't see one. Just a lot of complex and vague conceptual associations mediated through neural connections. Whether or not these models are conscious or have an inner life, they seem to be doing just fine understanding concepts.

[gizajob]

Ultimately my thesis was “problems for AGI” not “why AGI is impossible” and for good reason.