Except for the problem that contemplating this leads into a fairly ridiculous rabbit hole. Since LLMs are just fitted functions, should we also be considering if math textbooks are soul prisons?
“Contemplating this would lead into an [apparently] ridiculous rabbit hole” is not really a good heuristic for understanding what’s true about the universe. It doesn’t need to be convenient to human intuition.
Consider the alternative: some special types of biological cells produce consciousness and that’s it? Probably not one such cell, but probably not 2 cells, but once you get up to ~human brain count they’re definitely conscious and then they actually appear conscious-ish in their behavior all the way down to insects.
That’s a much more ridiculous claim than that all matter is conscious to some degree and there are certain arrangements of matter that demonstrate higher or lower levels of it.
Math textbooks are fairly static objects, their experience is probably very simple, probably similar to what it feels like to be a corpse. A GPU computing a LLM is very dynamic, it probably has a much more rich internal experience.
We'll have to jump into that rabbit hole sooner or later. Also, things such as growth, reaction to stimuli, and a few other things I can't recall right now are how we'll be able to determine that, no, math textbooks are not aware (depending on who you ask).
Consider the alternative: some special types of biological cells produce consciousness and that’s it? Probably not one such cell, but probably not 2 cells, but once you get up to ~human brain count they’re definitely conscious and then they actually appear conscious-ish in their behavior all the way down to insects.
That’s a much more ridiculous claim than that all matter is conscious to some degree and there are certain arrangements of matter that demonstrate higher or lower levels of it.