[zodzedzi]

> Generally, the models are compressing their understanding of all text, and in doing so, they're learning high order concepts

Are these higher order concepts accessible to us? E.g. can we list those learned concepts?

(Re-reading the paper you linked now...)

[tikkun]

My understanding is that the answer is generally: not yet.

(I wish, I suspect we'll be able to learn some interesting things about the universe, about humans, and so on, by seeing what LLMs found to be highly explanatory / high order concepts)