[laichzeit0]

I don’t actually even know what people are hinting at when they say that LLMs replace the need for building custom models. Regression models? People are using LLMs instead of say building a Bayesian hierarchical model? That’s not possible. Time series modeling using an LLM? Also ridiculous. Recommender systems? Ok maybe, still utterly ridiculous and abysmally slow.

For anything NLP sure, it definitely wins. However, I’ve just recently used some big fancy OpenAI model to actually just label thousands of text data for me, just so I could build a classifier with CatBoost. Guess what, inference speed is at a guaranteed sub 100ms and it costs $0 in tokens. The”AI Engineer” solution here would be just run every classification request through an LLM.

AI Engineering is going to have the same problem we had when Data Science as a term arrived and you had every Statistician saying they’re just re-inventing everything that exists in statistics, poorly.

[jappgar]

You're right. For years the real impediment to "AI" products at many companies was the sheer crappiness of ML frameworks which were built by and for grad students, not professional engineers.

When LLMs appeared it was just so much easier to use then as an uber model and leave behind the training and inference infrastructure (if you can even call it that).

Now that LLMs can code I expect we'll be coding up custom model pipelines more and more... but only when we stop subsidizing LLMs.