[avg_dev]

thank you for this. i am not well up on consciousness (or machine learning) and i have seen chatbots/llms hallucinate and such, and i have also seen them do amazing things. i have wondered to myself a few times lately: how do i know that im dissimilar in nature from these things?

so i ask you a followup question: what are some easy to understand ways in which a human's thought process would differ from an llm's behavior?

also for anyone else wondering: https://www.merriam-webster.com/dictionary/qualia

[nathan_compton]

There are a lot. First of all, these LLMs do not learn in-situ. They are entirely static (apart from the prompt). To teach an LLM something new is an ex-situ process, more or less totally unrelated to the way it predicts. Contrast that with a brain: brains are constantly learning (in fact, it is difficult to imagine how a brain as we understand it could work without constantly learning).

In a related way, because we learn on-line and constantly, our brains have to also maintain goals, rewards and punishments, etc etc. We have neurons for all of the trivia of keeping us moving, seeking new input, generalizing it, throwing away bad information, etc. For an LLM all of that is external. The LLM doesn't have any reason to even distinguish between generation and training. All the weight updates are calculated by a (relatively simple) external process. Furthermore, LLMs are entirely _feed forward_. The input comes in, a lot of numbers are crunched, and then output comes out. There is no rumination (again, the analogy for rumination in an LLM is in the training process, which is not embodied in the LLM).

Much of the content of our consciousness is perceptions relating to all of these things. I think its possible that artificial neural networks may one day do enough of these things that I would admit they are conscious, but architecturally and fundamentally, I don't see any reason that an LLM would have them.

I also don't think even GPT4 is that intelligent (fantastic recall, though). It does an impression of a cognitive process (literally by printing out steps) but that doesn't seem compelling enough for me to imagine a theory of mind underneath. A model of text, sure, but not a mind.

[avg_dev]

Wow, thanks again!

I really enjoyed this response and I have learned from it. (Which I guess an LLM could not do while generating something!)

Some references I glanced at (I mostly read the top paras):

- https://en.wikipedia.org/wiki/In_situ

- https://en.wikipedia.org/wiki/Theory_of_mind

Really enjoyed this response, and feel like I've developed a better understanding of some of the concepts relating to generative ML as it is used in LLMs.

An aside: I took a course on ML in a university a few years back, and it was interesting (it was an intro and survey course offered by the CompSci faculty), but difficult for me. I excelled at implementing using Keras/TF code in Python, and I had fun manually implementing some gradient descent algorithm but a lot of the math including all of the multi-var calc, stats, probability was quite difficult for me to wrap my head around, and I really didn't feel like I got a solid grounding on a meta-level of what we were doing or why. I have been reading a bit about LLMs and I think your post has filled in some of the gaps in what at this point I was really looking to understand.