[jen729w]

This question makes no sense (logically, not grammatically) in the context of LLMs.

They don't 'know' anything. They are a many-dimensional matrix of the next most likely syllable given all syllables that have come before (roughly speaking).

To ask what it 'knows' is to ask why a chicken crossed the road.