[nojvek]

It's just assumed that we need to make models bigger to get more out of them. That is from observation, no one has actually distilled different kinds of intelligences from an LLM to say this is the minimal model size needed for this kind of problem.

Examples: Databases scale with data stored in them. It's a linear relationship. We have databases that load pages on demand as needed and keep hot ones in cache.

There is no 'database page' alternative for LLM's yet. Someday there will be and the paradigm shift will happen.

Local 100 tok/s+ is a gamechanger for sure.