[catlifeonmars]

Why does a language model have to be monolithic? I think retraining a model is expensive (relatively speaking). Is there some way to bolt on specialization?

[ambewas]

That's exactly the issue. Retraining is too expensive & needs too much iteration to work efficiently I think.