[laborcontract]

The larger models generate high quality textbook-like synthetic data which is used to develop the model's reasoning skills. Microsoft's Phi series is a demonstration of this. These models do not have the ability to absorb and retain a lot of factual knowledge due to the low parameter count. However, they do have the ability to reason as well as larger models, which means these models perform best when most of the factual stuff is provided in context.