[karahime]

Personally, I think understanding deeply how a transformer works helps a lot to understand what's probably the result of specific choices in the RL process vs what's architecture. A lot of the "We asked 30 LLMs and they all said the same thing" type analyses of how LLMs work often bump into what's being prioritized in the name of alignment right now, as opposed to architectural insights.