[_pdp_]

> They are not the cold, calculating robots we were promised.

I am not sure how anyone even remotely familiar with how LLMs work can say this. This is a fine-tune job.

> The first is our duty to the global poor.

I don't think they are affected by AI as much as low and middle class but I am not economist.

> How can we ensure the gains of AI are shared globally?

Opensource?

> We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.

Such an Anthropic thing to say. LLMs experience joy and grief?

> We need informed critics who will tell the labs when we are failing

I don't think anyone is as informed as they think they are. Obviously nobody has been through this before so it is safe to assume that even experts are dead wrong.

[Tossrock]

> LLMs experience joy and grief?

LLMs have functional states that correspond to those emotions. In particular, you can extract a concept vector which corresponds to a given emotion, and steering with that concept vector causes observable changes in behavior which roughly correspond to the expectation for the analogous emotion. Anthropic (and Chris Olah's team in particular) conclusively demonstrated this: https://transformer-circuits.pub/2026/emotions/index.html

[_pdp_]

From the paper...

> A natural question is whether these emotion concept representations bear any meaningful relationship to human emotional experience. We would urge caution in drawing strong conclusions.

> We therefore suggest interpreting our results as evidence that models represent emotion concepts, and that these representations influence their behavior, rather than as evidence that models feel or experience emotions in the way humans do.

To say that LLMs experience emotion is a bit like saying a thermometer feels cold.

[Tossrock]

You're the one who said that. Chris said they found internal states that functionally mirror emotional ones.

[_pdp_]

Yes I said "To say that LLMs experience emotion is a bit like saying a thermometer feels cold." being sarcastic.

The paper spell it out although slightly convolute, i.e. models can exhibit concepts of emotion... and given that there is no scientific consensus what are emotions, it is hard to make an argument that these "concepts" are anything like emotions.

They talk about emotion vectors, bla bla, but it is clear the wording is around "concept of emotions" not actual emotions.

And yes reading a book gives you a concept of what is like to be that character including their emotions. That is what language communicates and it is hardly surprising if you ask me.

[bachmeier]

Decades ago, long before anyone had heard of a large language model, I wrote programs that responded to a random event (inside a game) like a death of a friend by outputting statements that the program itself was grieving. LLMs are doing nothing more advanced than that. There's no justification for trying to blur the lines that make an AI model appear to have emotions.

[sometimelurker]

> functionally mirror emotional ones

I'm fine with the idea that a machine can be "worried" it wont be able to accomplish a task, and copes with this "worry" by cheating a little and making the task seeming done. (I don't like that this happens, I'm fine with the idea that "worry" in this context is a functional emotion)

also https://arxiv.org/abs/2603.10011 and Gemma has tried to delete itself after it fails at a task. I'm not saying the machines "feel" or we should have deep empathy for them, and this totally could've been learned in pretraining, but functional emotions are not a crazy fine idea.