[skybrian]

I’m a little skeptical of processes that seem to create more information than you had to start with. For a game like chess or Go, it makes sense, because winning strategies are implicit in the rules of the game, but it takes a lot of computation to discover the consequences. Similarly for math where theorems are non-obvious consequences of axioms. And computer code can be similar to math.

But how does that work for an LLM in general? They’re trained on everybody’s opinions all at once, both right and wrong answers. They’re trained to generate text supporting all sides of every argument. What does more training on derived text actually do?

[laborcontract]

The larger models generate high quality textbook-like synthetic data which is used to develop the model's reasoning skills. Microsoft's Phi series is a demonstration of this. These models do not have the ability to absorb and retain a lot of factual knowledge due to the low parameter count. However, they do have the ability to reason as well as larger models, which means these models perform best when most of the factual stuff is provided in context.