[readyplayeremma]

This is a good overview of a number of emergent things that remain unexplained about the capabilities of some large language models:

https://www.scientificamerican.com/article/how-ai-knows-thin...

You can follow the multitude of things from there to the papers published that describe each of the cases in greater depth.