[margorczynski]

But that's not an NLP problem at heart. Language is just a collections of tokens (words, letter) that are tied together by certain rules to convey some meaning. There is no concept of reality per se.

For example, consider filling the blank:

A giant ______ flew over my head!

It can be a plane. Or a dragon. Or an UFO. Or a balloon. The thing is all of those are correct answers language-wise and the model works correctly as long as what gets filled in conforms to the rules of the given language.

The language that we generate encodes reality to some extent and the model picks up those correlations but there is no concept of reasoning or reality behind it. Maybe it is emergent at some point (as to effectively compress it needs to encode some subset of rules governing our reality) but it is not an agent that optimizes for understanding our reality. Something like Dreamer would be much closer to that.