[altruios]

The only issue. *

Reasoning is hard, reasoning about colors while wearing glasses that obfuscate the real colors... even harder... but not the core issue if your brain not wired correctly to reason.

I suspect the way out of this is to separate knowledge from reason: to train reasoning with zero knowledge and zero language... and then to train language on top of a pre-trained-for-reasoning model.

[onoesworkacct]

LLMs already use mixture of experts models, if you ensure the neurons are all glued together then (i think) you train language and reason simultaneously