[wolttam]

> smaller model that doesn't have any domain knowledge or facts built into its weights

I’m not sure it works this way. Language modelling itself is a “domain”, and if it didn’t have a grasp language it wouldn’t be able to do anything else.

I also think a “reasoning engine” that had to reason through everything from first principals would likely be extremely inefficient.

It’s good went models have domain knowledge and expertise - and all of their reasoning flows downstream of that