[Muhammad523]

I think many people have. That is, in my opinion, because of all the anthropomizing (sorry for typos!) language used. The companies building these systems keep calling their newest features after human processes, for example "Dreaming", "Thinking", and the fact that they make their models talk in first person

> Wait, I noticed a pattern in my previous responses: I had some weird typos/letter additions ('sgreat', 'askinsg'). Actually, wait — did I do that on purpose or was it a glitch?

A person who has no idea what an LLM is would likely fall into this "trap"

[thepasch]

I know quite well what an LLM is and how it works! I've captured activation patterns and written scripts to analyze how they compare to one another in response to a set of controlled and curated prompts; in particular, trying to replicate the functional emotional vector findings from the Anthropic paper (https://transformer-circuits.pub/2026/emotions/index.html) on various open source models; successfully on some, less so on others. FWIW, Gemma 4 31B was among those where clear patterns did emerge.

What I don't know quite as much about is how cognition works in biological computers - and I suspect you know just as little as most of the rest of us do in that regard! So I think it's not entirely appropriate to make sweeping claims about what artificial neural networks, fundamentally, can and cannot do. Most of what we can do is poke and prod at them and see what happens, which is exactly what this piece is about.

[Muhammad523]

I see, this experiment is a fun thing to do. My comment wasn't concerned much on why we do this, but rather on the fact that many people are starting to see LLMs as genuine entities, and i really don't think they are (also, i feed bad about using the word "genuine" after it has been abused so badly)

[thepasch]

That's fair. FWIW I don't think they are either, but I specifically don't think they're fundamentally incapable of it, and I think that as models grow, we're going to see more and more concepts and behaviors emerge that might, one day, with enough parameters and enough training, approach the parts that a genuine entity requires to be a genuine entity. Whatever those are.

No idea if that's true or if there's some sort of "special sauce" required that you just can't get from artificial trained networks. But I've been a functionalist since long before LLMs emerged, so the signs of these behaviors that we are already seeing in the models of today aren't very surprising to me!