[gostsamo]

You can improve results with cleaner datasets and if you prioritise a certain goal like conversation, code compleation or reasoning.

[andai]

I'm reading the Textbooks Are All You Need paper, which goes into this idea. The result of that research was Phi 1, and eventually Phi 3 (released a few days ago).