[legosexmagic]

> Nobody knows how even the current generation of LLMs work (at the plumbing layer, sure, but the high-level behavior is a complete mystery).

i dont think thats fair to say tbh. clearly language has alot of patterns in it. so it should be possible to come up with a network that finds them. if the loss converges to something reasonable and the amount of training data is greater than what the network is able to memorize, then the only possibility is that the network learned whatever general patterns the data happens to have.

[anonzzzies]

The emergent behaviours are not really understood from that context.