[scotty79]

I can't really blame them that the biggest labs focused on trainig and realeasing huge models.

The niche for small models should be filled with medium sized labs doing distillations of the huge ones into consumer grade hardware runnable models and LORAs for the huge ones.

[docheinestages]

I think AI will evolve the same way computers did. We're somewhere in the 80s-90s timeline of the evolution. My prediction is that on-device models will have excellent tool-calling, reasoning, and general skills, but the domain-specific knowledge will be retrieved on-demand from vendors like Google. Rather than downloading models, each device will have a hardware component with weights baked into silicon for maximum efficiency.

[gunalx]

Weigths directly in silicon is a bad idea with the way the space is pacing. Just look at chatjimmy.ai it is fast, but on the once good llama3.1-8b but now pretty useless.

[docheinestages]

That's why I believe the baked weights should represent a generic model that is good at general thinking and tool calling. The rest can be augmented by the OS.