[iainctduncan]

I work in tech diligence so I look at companies in detail. I have seen a couple where good machine learning is going to make a massive difference (whether it will keep them ahead of everyone is a separate question). I think it really boils down to:

"Is this a problem where an answer that is mostly right and sometimes wrong is still a great value proposition?"

This is what people don't get. If sometimes the answer is (catastrophically) wrong, and the cost of this is high, there's no market fit. So I think a lot of these early LLM related startups are going to be trainwrecks because they haven't figured this out. If the cost of an error is very high in your business, and human checking is what you are trying to avoid, these are not nearly as helpful.

I looked at one company in this scenario and they were dying. Couldn't get big customers to commit because the product was just not worth it if it couldn't be reliably right on something that a human was never going to get wrong (can't say what it was, NDAs and all that.) I also looked at one where they were doing very well because an answer that was usually close would save workers tons of time, and the nature of the biz was that eliminating the human verification step would make no sense anyway. Let's just say it was in a very onerous search problem, and it was trivial for the searcher to say "wrong wrong wrong, RIGHT, phew that saved me hours!". And that saving was going to add up to very significant cash.

So killer apps are going to be out there. But I agree that there is massive overhype and it's not all of them! (or even many!)

[myownpetard]

Interesting final point. It's like the business equivalent of NP-Complete problems, difficult to compute but easy to verify.

Can you give any examples of those types of problems you've encountered?

[killernap]

That's interesting. Quite the needle to thread. I wonder how big the market will be for niche models that aren't commodities.

It needs to be something lucrative enough that training the model is not-trivial but not so lucrative Microsoft/Google would care enough to go after. And it somehow needs to stay in that sweet spot even as Nvidia chips away at that moat with each new hardware generation.