[Mike_12345]

> I am all about emergent properties of neural networks, but I absolutely do not believe that LLMs have them, specifically because of the way they are designed.

Even when faced with evidence that contradicts your beliefs and proves that you are wrong?

LLMs are a type of neural network. What fundamentally prevents LLMs from having emergent abilities while other neural networks do have them?

How do you explain the emergent abilities that we have actually observed in LLMs?

[PaulDavisThe1st]

You don't have any evidence, and neither does any paper I've read claiming as such.

The level of naievete around this stuff is quite incredible.

BEFORE: "We have no idea really what a system like this will do"

... amazing results ...

AFTER: "Wow, look at those emergent properties"

[Mike_12345]

Here is an example of an emergent ability of ChatGPT that you can try yourself right now.

Give it this prompt: "Write a short play where "Karen" (who behaves as the Karen meme) is on a romantic date with Hunter S. Thompson. They are eating at a Chinese restaurant."

It is able to script their interaction and dialog in a way that makes sense in the context of the setting and their personalities, including an absurd meme character. That is an emergent ability that it was not trained to do and is certainly not in its training set.

Try it. Add different characters. Ask it to rewrite the play using pirate metaphors. You can go deep into its "mind" and see the emergent abilities at play. Just apply some creativity and skip the boring arithmetic problems, as that's a well accepted weakness of this type of model.

[Mike_12345]

What do you mean we don't have evidence? It has already been presented to you. You choose to reject it for reasons I truly don't understand. Search on Google Scholar if you want a more academic explanation. You can try it out on ChatGPT right now and see it yourself.

> The level of naievete around this stuff is quite incredible.

"Emergent properties" has a formal definition in the literature.

In machine learning we don't explicitly program the machine to understand anything. It automatically learns patterns in the data. That's the entire point of machine learning. With such a large neural network and training set obviously it's hard to predict all of its capabilities due to the sheer scale of it all. Of course we cannot predict exactly how it will model things.

Take this for example. No one programmed it to understand Go. It learned by itself and became the world champion. That's what deep learning is capable of.

https://en.wikipedia.org/wiki/AlphaGo_Zero

"The neural network initially knew nothing about Go beyond the rules."