Hacker News new | ask | show | jobs
by xg15 71 days ago
I think a counterargument would be parallel evolution: There are various examples in nature, where a certain feature evolved independently several times, without any genetic connection - from what I understand, we believe because the evolutionary pressures were similar.

One obvious example would be wings, where you have several different strategies - feathers, insect wings, bat-like wings, etc - that have similar functionality and employ the same physical principles, but are "implemented" vastly differently.

You have similar examples in brains, where e.g. corvids are capable of various cognitive feats that would involve the neocortex in human brains - only their brains don't have a neocortex. Instead they seem to use certain other brain regions for that, which don't have an equivalent in humans.

Nevertheless it's possible to communicate with corvids.

So this makes me wonder if a different "implementation" always necessarily means the results are incomparable.

In the interest of falsifiability, what behavior or internal structures in LLMs would be enough to be convincing that they are "real" emotions?

2 comments

I believe that it is possible to make an artificial system that can have emotions in a way that cannot be meaningfully discriminated from those of an animal or a human.

However, I believe that designing such a system would be pointless and very wrong.

While emotions are something that is normally associated with a physical system that encounters in the real world various helpful or harmful experiences, one could make a program that simulates completely such a physical systems with emotions; as it lives in a simulated world, and then one could say that this program has emotions.

On the other hand, unlike with the kind of program that I have mentioned before, I do not agree that an LLM has emotions, but only that it mimics human emotions, as they had been recorded in the training texts.

There is no component of an LLM that can intrinsically generate emotions. An LLM that is trained only on texts without emotions, e.g. on program sources stripped of comments, will not show any emotion whatsoever, regardless of what you put in its input prompt.

On the other hand, when you train an LLM on texts that record human emotions, then whenever the LLM input contains something that is similar to what has elicited the human emotions recorded in the training texts, then the LLM will output a token probability distribution that will generate a response similar to the reactions of humans. Unlike a book or an audio or video recording, the output of the LLM usually will not match exactly one human emotion recording, but it will mix many of those recorded, but it will still be limited by the content used for training.

"Parallel" evolution is just different branches of the same evolutionary tree. The most distantly related naturally evolved lifeforms are more similar to each other than an LLM is to a human. The LLM did not evolve at all.
Evolution is the way how the "mechanism" came to be, which is indeed very different. But the mechanism itself - spiking neurons and neurotransmitters on one hand vs matrix multiplications and nonlinear functions (both "inspired" by our understanding of neurons) don't seem so different, at least not on a fundamental level.

What is different for sure is the time dimension: Biological brains are continuous and persistent, while LLMs only "think" in the space between two tokens, and the entire state that is persisted is the context window.

> The LLM did not evolve at all.

Evolution and Transormer training are 'just' different optimization algorithms. Different optimizers obviously can produce very comparable results given comparable constraints.

The training process shares a lot of high-level properties with the biological evolution.
"Minimize training loss while isolated from the environment" is not at all similar to "maximize replication of genes while physically interacting with the environment". Any human-like behavior observed from LLMs is built on such fundamentally alien foundations that it can only be unreliable mimicry.
The environment for the model is its dataset and training algorithms. It's literally a model of it, in the same sense we are models of our physical (and social) environment. Human-like behavior is of course too specific, but highest level things like staged learning (pretraining/posttraining/in-context learning) and evolutionary/algorithmic pressure are similar enough to draw certain parallels, especially when LLM's data is proxying our environment to an extent. In this sense the GP is right.