[ben_w]

This feels correct.

Without speculating on the internal mechanisms which may be different, what surprises me the most is how often LLMs manage to have the same kind of failure modes as humans; in this case, being primed as "bad" makes them perform worse.

See also "Stereotype Susceptibility: Identity Salience and Shifts in Quantitative Performance" Shih, Pittinsky, and Ambady (1999), in which Asian American women were primed with either their Asian identity (stereotyped with high math ability), or female identity (stereotyped with low math ability), or not at all as a control group, before a maths test. Of the three, Asian-primed participants performed best on the math test, female-primed participants performed worst.

And this replication that shows it needs awareness of the stereotypes to have this effect: https://psycnet.apa.org/fulltext/2014-20922-008.html

[Eisenstein]

I'm curious why you find it surprising?

In my view, language is one of the basic structures by which humans conceptualize the world, and its form and nuance often affect how a particular culture thinks about things. It is often said that learning a new language can reframe or expand your world view.

Thus it seems natural that a system which was fed human language until it was able to communicate in human language (regardless of any views of LLMs in an greater sense, they do communicate using language) would take on the attributes of humans in at least a broad sense.

[ChrisMarshallNY]

> It is often said that learning a new language can reframe or expand your world view.

That was sort of the whole concept of Arrival; but in an even more extreme way.

[Eisenstein]

Learning an alien language allowed people to disconnect their consciousness from linear time, allowing them to do things in the past with knowledge they gained later, though I seem to recall they didn't know why they did it at the time, or how they had gotten that information.

[ChrisMarshallNY]

I'll have to watch it again. I suffer from C.R.A.F.T. (Can't Remember A Fucking Thing).

[ben_w]

Lois Lane and Hawkeye play Pictionary with a squid inside one segment from a fleet made out a massive Terry's Chocolate Orange, each part of which is hovering over a different part of the world in exactly the way real chocolate oranges don't.

[ben_w]

It's surprising, because only leading-edge V[ision]LMs are of comparable parameter count to just the parts of the human brain that handle language (i.e. alone and not also vision), and I expect human competence in skills to involve bits of the brain that are not just language or vision.