[AlexandrB]

There's another case for pessimism as well: cost. It's possible that many AI applications aren't worth the money required for the extra compute. AI-enhanced search comes to mind here: how is Microsoft going to monetize users of Copilot in Bing to justify the extra cost? Right now a lot of this stuff is heavily subsidized by VCs or the MSFTs of the world, but when it comes time to make a profit we'll see what actually sticks around.

[_factor]

Better question: why does a simple search for “What color is a labrador retriever” require any compute time when the answer can be cached? This is a simple example, but 90% of my searches don’t require an llm to process a simple question.

[jazzyjackson]

One time I came across a git repo that let me download a gigabyte of prime numbers and I thought to myself, is that more or less efficient than me running a program locally to generate a gigabyte of prime numbers?

The compute for a direct answer like that is fractions of a penny, it might be better to create answers on the fly than store an index of every question anyone has asked (well, that's essentially what the weights are after all)

[_factor]

It’s in interesting question. I assume they’re using accelerators, and the alternative is a disk or memory hit. It still seems expensive to me.

https://www.linkedin.com/pulse/rising-cost-llm-based-search-...

[jacobr1]

This seems true as far as incentives go. But how much of that cost driver will be due to efficiencies driven by companies like NVIDIA? They seem well poised to benefit from a lot of the increased (non-hype) use of AI. Seems like we spent a decade or more of stalled CPU performance gains chasing better energy efficiency in the data center, same story could play out here.