[goatlover]

It's the large training data which contains the knowledge for that complex and useful emergent behavior. It's like if you could import vast information about the world into Conway's game of life to enable increasingly complex levels of emergence.

[elcritch]

Oh idea of "vast patterns" in Conway's game of life seems like a great metaphor for LLM transformers. Explaining hidden markov chains with some extras or whatnot to a layman doesn't give a good mental image but is what I guess it's like.

Probability chains alone don't seem to give a good mental visualization of how such a system comes to certain "decisions" or "thought patterns". But watching the Game of Life you can see fascinating patterns which emerge and lead to interesting patterns. That's easy to extrapolate.

Maybe in the future NNs will be understable sorta like Game of Life, "oh that NN section is running pattern 27 on XY input data. That's gonna be an unstable behavioral element combined with pattern 38c over here." Not sure if that's a fascinating or dreadfully boring prospect though.