[jonash54]

Producing specific niche models for 100 year old industries that have mountains of data and warehouses full of folders will be the european take on AI.

It may come late but it‘ll be safe and reliable. It also requires a lot of OCR.

[lumost]

The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)

[berkes]

The General Models' business-model is also looking more weak every iteration.

Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page".

Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing under $10 for thousands of such classes and functions.

Or the, over ten dollars worth of tokens when some "agent" using a general model, tries to perform the task I gave it to "read the event on example.com/event/1337 and put it in my calendar", include commute time as well"

The "general models" currently only become smarter by growing bigger and having larger context windows - by becoming exponentially more expensive to train and to run and to interact with. Whereas "Niche" models can do the things that "normal code" cannot do, and improve by tuning and tweaking only that. Their goal is then to fill in gaps that traditionally are hard or impossible with normal software. Wheras the goal of a general model (with agentic reasoning)is to replace that entire "normal software".

One example: I am not interested in "chatting with my calendar". I'm interested in a calendar because it is a well known view (UI) of my planning and tasks, but I see a lot of opportunities where AI can improve my working with this calendar. I may be interested in a smarter screen when I hit "+ Add event"; one that has knowledge of my previous events and patterns (some RAG vector db maybe). One that maybe has access to content I just copied, or read (though: privacy?) or can open my camera to let me shoot a pic of something that has the event info on it. In such a set-up, Niche LLMs perform dedicated tasks: determine patterns (he always books a Yoga class on wednesday or thursday, two days in advance, so lets suggest a yoga class), determine existing content (event is planned 100Km from his home, so lets suggest the commute based on previous commutes like this). Or an OCR model. Or an autocomplete model. Relatively simple, niche models, called from within software to aid me when "calendaring". Not replace the entire calendar with some chat.

¹Edit: This was a rather unscientific research of mine, where I compared some models to read from photographs, compared purely on costs and timing. "Opus" or other generic LLMS with image input capabilities commonly did better on "performance" esp with difficult input such as a picture of a poster of some rock event.

[lumost]

the counter point is that building or selecting the specialized model may cost as much as the lifetime inference costs of the task with the specialized model.

If I need to pay someone 300k to make the model and infrastructure... then I would need to process many documents to recoup my OCR costs compared to asking claude code nicely.

Perhaps the model zoo is becoming good enough that the cost to find a specialized model is not so high?

[berkes]

> If I need to pay someone 300k to make the model and infrastructure

I was arguing for the existing AI-companies that already make and offer niche models. Like Mistral. But AFAICS, all AI companies have and offer such models.

So all you need to do, is use the existing models. And, yes, select it.

Which, ironically, I would highly value as a niche model myself. I spend way too much time following the breakneck race of the various companies just to pick the right models for my tasks at hand. "Just pick the latest" often yields worse results, or is magnitudes more expensive, or significantly slower, or all of it. "Just pick the most popular" can prove expensive, inefficient for some task etc. This investment, also ironically, has proven something of a "moat" for me. I know very well what Mistral and Anthropic offer. So I won't even bother with OpenAI, Google, X, Tencent etc etc etc models. I just don't have the time to keep researching the latest offers for their pros and cons.

A model that acts as "decision maker" and as proxy, as a conductor, that directs and transforms my questions and sends them off to the right model, right tools, right MCP etc, would be very welcome for me. So that I can just pick that one, and have the highly dynamic world of LLMs and other models shift like undercurrent beneath the surface of this One Model To Rule Them all.

[notahacker]

Sure, but things like "OCR tool" or "agentic calendar assistant" or "agentic interface to CAD" don't feel like things I'd need pay 300k to build out custom infrastructure for, they feel like something lots of people/companies would rather pay an appropriate-sized subscription to a model optimized for that use case rather than burn ludicrous amounts on generalist models optimized for burning as many tokens as possible reasoning about how to update a calendar from first principles.

I mean, I also I think the "agentic calendar assistant" is more likely to come out of the Valley than Europe[1] and that the big AI startups will buy some of the niche AI startups gaining traction in strategic markets, but there are lots of niches that notionally benefit from transformer-based models

[1]probably a calendar assistant with a ludicrously inflated valuation based mostly on the assumption that Google/Meta/Anthropic will buy them to shut them down three years later...

[nolok]

The cost is getting worse and worse for large general models, they're already way past that point in economics. Also, mMistral specialize in "on site" models, not remote. In terms of capex, renting factory/warehouse/whatever robots versus buying them and depreciate has already been played out, companies didn't want to replace human employees with robots employees.

[cyberax]

We are making a niche model that we're now expanding. You'd be surprised how the general models suck for anything vision-related.

And even if you use all the tricks in the book to make them work for you, the cost can easily be 1000 _times_ more than the specialized model. Ditto for speed.

This is especially important for things like robotics or navigation.

[philipkglass]

It seems like a stronger story for robotics, since smaller models can always react to the environment faster than large models at a given hardware budget. Also because robots that keep their models local for latency or reliability aren't going to be carrying many kilowatts of inference capacity.

[lumost]

remote inference should be sufficient for most robotics applications with potentially a small model for safety critical actions running locally.

Unless you are in military robotics or automotive of course :)

[AlotOfReading]

There are many, many factories that still don't have internet access on the floor, and commercial inference generally has response latencies measured in seconds. I struggle to imagine a factory spending hundreds of thousands for the local compute to run a large model either, given how cheap they are about expenses.

I'm also skeptical that you can cleanly differentiate between "safety critical actions" and "actions", though this is less of a practical concern given how laissez-faire some manufacturers are. For context, I work on safety critical robotics (in automotive).

[baq]

I expect the bitter lesson to continue to be bitter. Mistral must at least attempt to catch up to SOTA 6 months ago.

[coredev_]

Do they really? "SOTA" is great for development and creating content but for industrial needs.... perhaps they are not really "SOTA"?

[baq]

It follows directly from the bitter lesson - a frontier model can be relatively cheaply distilled into anything you need to run quickly (and a frontier model like Mythos will help you distill it quickly), decidedly not true the other way around.

[esseph]

That's been the playbook for Palantir for a long time now. Forward Deployed Engineers at the customer, living and eating with them for 3+ months.