[kllrnohj]

Because the transformer architecture that enabled modern LLMs wasn't invented until 2017[1]?

1: That's the "T" in GPT fyi, even though Google is the author of the research paper that changed everything

[nonethewiser]

Right. So we had enough information to train LLMs but not the technology to build it.

So the initial models arent just distilled from information. We’ve always had the information.

[anigbrowl]

Yes, but not the secret of distillation.