[yojo]

Every time someone says “but dark fiber”, someone else has to point out that graphics cards are not infrastructure and depreciate at a much, much higher rate. I guess it’s my turn.

Fiber will remain a valuable asset until/unless some moron snaps it with a backhoe. And it costs almost nothing to operate.

Your data center full of H100s will wear out in 5 years. Any that don’t are still going to require substantial costs to run/may not be cost-competitive with whatever new higher performance card Nvidia releases next year.

[typewithrhythm]

How does the cost breakdown of all these new datacenters, cooling, and power delivery systems compare to the cost of the GPUs themselves?

There is a surprising amount of real long-term infrastructure being built beyond the quickly obsolete chips.

[BlarfMcFlarf]

Poorly. GPUs are easily the bulk of the costs, and a disposable asset.

[musebox35]

That is a fine point. However I am not sure if replacing the gpus themselves will be the bottleneck investment for datacenter costs. After all you have so much more infrastructure in a datacenter (cooling and networking). Plus custom chips like tpus might catch up at lower cost eventually. I think the bigger question is whether demand for compute will evaporate or not.

[wmf]

When the bubble pops the labs are going to stop the hero training runs and switch the gigawatt datacenters over to inference and then they're going to discover that milking existing GPUs is cheaper than replacing them.