[segasaturn]

That still means that that AI firms don't have to buy as many of Nvidia's chips, which is the whole thing that Nvidia's price was predicated on. FB, Google and Microsoft just had their their billions of dollars in Nvidia GPU capex blown out by $5M side-project. Tech firms are probably not going to be as generous shelling out whatever overinflated price Nvidia was asking for as they were a week ago.

[epicureanideal]

Although there’s the Jevon’s Paradox possibility that more efficient AI will drive even more demand for AI chips because more uses will be found for them. But possibly not super high end NVDA chips but instead little Apple iPhone AI cores or smartwatch AI cores, etc.

Although not all commodities will work like fossil fuels did in Jevon’s Paradox. It could be the case that demand for AI doesn’t grow fast enough to keep demand for chips as high as it was, as efficiency improves.

[talldayo]

> But possibly not super high end NVDA chips but instead little Apple iPhone AI cores or smartwatch AI cores, etc.

We tried that, though. NPUs are in all sorts of hardware, and it is entirely wasted silicon for most users, most of the time. They don't do LLM inference, they don't generate images, and they don't train models. Too weak to work, too specialized to be useful.

Nvidia "wins" by comparison because they don't specialize their hardware. The GPU is the NPU, and it's power scales with the size of GPU you own. The capability of a 0.75w NPU is rendered useless by the scale, capability and efficiency of a cluster of 600w dGPU clusters.

[alfalfasprout]

Wrong conclusion, IMO. This makes inference more cost effective which means self-hosting suddenly becomes more attractive to a wider share of the market.

GPUs will continue to be bought up as fast as fabs can spit them out.

[segasaturn]

The number of people interested in doing self-hosting for AI at the moment is a tiny, tiny percentage of enthusiast computer users, who indeed get to play with self-hosted LLMs on consumer hardware now.. but the promise of these AI companies is that LLMs will be the "next internet", or even the "next electricity" according to Sam Altman, all of which will run exclusively on Nvidia chips running in mega-datacenters, the promise of which was priced into Nvidia's share price as of last Friday. That appears on shaky ground now.

[alfalfasprout]

I'm not talking about enthusiastic computer users. To be frank, they're rather irrelevant here. I'm talking about companies.

[flowerlad]

> That still means that that AI firms don't have to buy as many of Nvidia's chips

Couldn’t you say that about Blackwell as well? Blackwell is 25x more energy-efficient for generative AI tasks and offer up to 2.5x faster AI training performance overall.

[Jlagreen]

And yet, Blackwell is sold out.

What does that tell us?

The industry is compute starved and that makes totally sense.

The tranformer model on which current LLMs are based on are 8 years old. But why took it so much time to get to the LLMs only 2 years ago?

Simple, Nvidia first had to push the compute at scale strongly. Try training GPT4 on Voltas from 2017. Good luck with that!

Current LLMs are possible thanks to the compute Nvidia has provided in the past decade. You could technically use 20 year old CPUs for LLMs but you might need to connect a billion of them.

[uxhacker]

It means personal ai on every computer. No privacy concerns, but saying that it is quite weird coming from a Chinese start up :)

[grogenaut]

It won't last long. Agents are where AI is going to go imho. That means giving the ai software access to the internet, and that means telemetry.

[windsignaling]

Always hilarious to see westerners concerned about privacy when it comes to China, yet not concerned at all about their own governments that know far more about you. Do they think some Chinese policeman is going to come to their door? Never heard of Snowden or the five eyes?

[Jlagreen]

The $5M was the cost of the training itself.

You can rent 10k H100 for 20 days with that money. Go and knock yourself out because that compute is probably higher than what DeepSeek received for that money. And that is public cloud pricing for single H100. I'm sure if you ask for 10k H100 you'll get them at half price so easily 40 days of training.

DeepSeek has fooled everyone by telling them that they need only so less money and people think that they only need to "buy" $5M worth of GPU but that's wrong. The money is the training costs of renting the GPU training hours.

Somebody had to install the 10k GPUs and that's paying $300M to Nvidia.

[chgs]

Imagine what you can do with all that Nvidia hardware using the deep mind techniques.