[npn]

fine tuning a small LLMs or even real small language models (like bert) is what the recommended way since the introduction of LLM. the benefit is pretty much obvious: faster to run, fully controlling the stack, better fit for the custom domain...

but in practice not many people do the fine tuning, for pretty much a single reason: large language models API cost are still very cheap, fast enough, and get improvement all the times. what the point of spend time (and money) to fine tune a specified model, then just when you release it a newer gen generic model is released and beat it?

but if someday the progress for LLM is slowed, or the price increased to the point calling api is not a viable approach any more, then surely the day of fine tuning and small models will come again.

[johsole]

I think we're going to get to that point on the 'S' curve. I also think the moat for a lot of companies is going to be their process and data, self hosting tuned models could be increasingly viewed as a trade secret.

[dimgl]

It's also really hard. And you need a lot of data for the fine tuning to produce good results.

[npn]

well usually you can just generate the data using LLM. use 2 or 3 different frontier models from different providers, then compare the results and pick the consensus. yes it is not 100% correct like when you make it manually, but then again even human makes mistakes.