[tsunamifury]

This shows how the individual weights and vectors work but unless I’m missing something doesn’t quite illustrate yet how higher order vectors are created at the sentence and paragraph level. This might be an emergent property within this system though so it’s hard to “illustrate”. how all of this ends up with a world simulation needs to be understood better and I hope this advances further.

[visarga]

The deeper you go, the higher the order. It's what attention does at each layer, makes information circulate.

[tsunamifury]

Thanks, I assumed that was the case, but they didn't make that explicit. Then the question is, is the world simulation running at the highest order attention layer or is it an emergent property of the interaction cycle between the attention layers.