[broken_clock]

Aren't there already a ton of startups doing finetunes for their local niche? Many aren't even "AI" companies - it's pretty easy to slap a finetune together if you enough data.

If you mean developing a model from scratch just for your niche - the bitter lesson is that scale is everything and that a finetune from an internet-scale model will outperform you easily.

[coffeebeqn]

DeepSeek has some something pretty remarkable. It’s certainly not “just” fine-tuning a Llama or a GPT prompt. More of a order of magnitude optimization