[LogicFailsMe]

"Those excited about generative AI are either the victim or the perpetrator of a con centered around a technology to ingratiate at the highest cost possible."

How else would you read the above statement? He's just preaching to his own choir IMO.

My take: like any gold rush, a lot of dumb ideas will get backed and they will all fail. And then we'll keep the ones that worked. SSND. Good luck picking the winners a priori.

[mlyle]

I read it in context as being about the market prospects of genai.

The problem is, when there is so much overinvestment, everything gets wrecked. In the aftermath of the dotcom boom there was at least a bedrock of fiber and still useful equipment to build upon amid the rubble. This time we are going so much further; also many of the durable assets are misplaced bets and the depreciating ones will depreciate more steeply.

[LogicFailsMe]

Someone should do the analysis of a decade and a half of Nvidia datacenter GPUs from Fermi to Kepler to Maxwell to Pascal to Volta to (Turing) to Ampere to Hopper to Blackwell and generate some hard depreciation numbers. Fiddling around a bit, 16-20% annual depreciation (so 5-6 years total and then any further revenue is bonus goods) it would appear, but that's a fiddle number.

But confounding this, K80s and V100s are still offered by cloud providers 13 and 9 years after their releases and academia still loves their GTX 1080 Pascals in their desktops. At companies, the beancounters take a computation and find the best architecture !/$ for that calculation. It does not need to be brand new shiny. It's Nvidia's job to make that case, not them. But anyway, the real data is right there. And those old GPUs demonstrate the dark fiber is already in place (and it's not so dark or they'd pull their racks).

AI is the special case. New GPU generations are the only way to access HW implementations of last year's research on precision modes and matrix math. If that slows down, that would be the first real bellwether of a slowdown. It hasn't happened yet. I'm a little surprised myself, but I also think coding agents are the vanguard of general design agents and that's going to hit a lot of industries at once. So as long as the next generation of GPU halves the price of tokens and doubles throughput (or better), the demand for tokens will continue to rise IMO.

What I don't think is that AI can come for anyone's job successfully no matter what the C-suite sorts insist.

In summary, if you're a bear, you can point to the depreciation cycle and scream the sky is falling. And if you're a bull you can point to GPUs staying in production for a very long time despite the depreciation. Guess we have to wait for 2030.

[sumeno]

5-6 years is wildly optimistic for GPUs in an AI data center

Try 1-2: https://www.tomshardware.com/pc-components/gpus/datacenter-g...

[LogicFailsMe]

Sure, according to an unnamed "GenAI principal at Alphabet" of whom "We could not verify the name of the person who describes themselves as 'GenAI principal architect at Alphabet' and therefore we cannot 100% trust their claims."

But let's run with Deep Layer(tm)'s hot take from 2024, GPUs all die in 2 years, no exceptions*. Poor guy just spent $400,000 on a DGX with 8 B200s, each of those B200s generates a piddly ~$3,000 in profit monthly spewing tokens, netting $576K in 2 years, that's a pathetic 20% annual return. Oh no... Won't someone call Michael Burry!

*Never mind the 3-year warranty or any extended service contract, that GPU is D E D and you're S O L.

[mlyle]

You gotta include 50k+ in power plus other expenses. Still looks like an ok return on capital, but you are betting heavily that cost of computing doesn’t fall much.

If token prices fall a bunch then it may not even be worth leaving on, depending on your facility’s relative power, cooling costs.

If we push far into oversupply eventually a bunch of firms building this infrastructure are going to lose out.

[scubbo]

SSND?

[tempfile]

same shit new day, I'm guessing

[scubbo]

Thank you