[grumpopotamus]

We have absolutely no idea what makes something conscious or how to measure whether something is conscious or not. It is perfectly reasonable to consider that AGIs might have conscious experience, and to discuss the moral consequences of that. Some of the leading researchers including Geoffrey Hinton, and some of the leading philosophers of consciousness including David Chalmers have expressed openness to the idea that LLMs may have conscious experience.

[anon373839]

Except for the problem that contemplating this leads into a fairly ridiculous rabbit hole. Since LLMs are just fitted functions, should we also be considering if math textbooks are soul prisons?

[ethanbond]

“Contemplating this would lead into an [apparently] ridiculous rabbit hole” is not really a good heuristic for understanding what’s true about the universe. It doesn’t need to be convenient to human intuition.

Consider the alternative: some special types of biological cells produce consciousness and that’s it? Probably not one such cell, but probably not 2 cells, but once you get up to ~human brain count they’re definitely conscious and then they actually appear conscious-ish in their behavior all the way down to insects.

That’s a much more ridiculous claim than that all matter is conscious to some degree and there are certain arrangements of matter that demonstrate higher or lower levels of it.

[anon373839]

I’m not making that claim either. My point is that it just isn’t a very productive area of inquiry (unless one derives some enjoyment from it).

[CuriouslyC]

Math textbooks are fairly static objects, their experience is probably very simple, probably similar to what it feels like to be a corpse. A GPU computing a LLM is very dynamic, it probably has a much more rich internal experience.

[dailyanchovy]

Math books are only static on your size scale

[herewego]

This is a bit of a reductionist viewpoint. Perhaps you’re disregarding the emergent properties of fitted functions at scale.

[notaustinpowers]

We'll have to jump into that rabbit hole sooner or later. Also, things such as growth, reaction to stimuli, and a few other things I can't recall right now are how we'll be able to determine that, no, math textbooks are not aware (depending on who you ask).

[js8]

> should we also be considering if math textbooks are soul prisons?

Why not? There are even philosophical theories that the reality itself is just a manifestation of mathematical laws.

[lm28469]

Yeah that's called arguments from authority, not so long ago the same people were mixing random crystals and dirt to make gold and trying to convince you that animals were automatons...

It's a bunch of GPUs running code... simpletons get deceived because it is displayed through a chat.

[ethanbond]

If you have some defensible definition of when/where/how consciousness emerges, you should publish a paper and go win your Nobel.

This is an unsolved problem.

[lm28469]

Right, but then again the burden of proof isn't on my side

[ethanbond]

What about the burden of proof that you are conscious? After all, you’re just a bunch of neurons firing signals, and 100% of your behavior is readily explained as an automaton in a meat machine.

[lm28469]

Do I need to prove my own consciousness to think that chatgpt isn't conscious ?

I swear people defending this idea would argue a plane and a bird are the same thing because "hey look, they both fly and have wings"

[herewego]

You have to prove your understanding of consciousness to be certain that LLMs are not conscious. It’s your certainty that I suspect most are reacting to here.

[ethanbond]

To have a sufficiently well-founded opinion to call people who disagree with you "simpletons," I'd argue that you indeed should have a defensible idea of what's conscious and what's not. Of course nothing is stopping anyone from being an asshole on the Internet so no, you don't "need" to do anything at all.

[CuriouslyC]

TBF ChatGPT is more alive and has more soul than some humans I've met.

[lm28469]

And Star Wars is more entertaining than my life, it doesn't make it a real story, if anything it's a testament to how alienating modern life became