[jpdus]

What's your opinion on the coming (or not) GPU crunch?

When looking at cloud GPU availability and current trends (no one except some enthusiasts and bigtech is finetuning and serving on a large scale yet and results keep getting better and better), I fear we will run into a situation where GPUs will be extremely expensive and hard to come by until supply catches up?

I ordered a high end PC with 4090 for the first time in years (normally would always prefer cloud even if more expensive) because I want to be on the safe side. What do you think, is this irrational and just a bubble thing?

[panarky]

It's a temporary crunch while TSMC expands capacity. They were reluctant to invest in capacity during the crypto crunch because they (rightly) understood that to be a temporary spike in demand. This time, they (also rightly) recognize AI as a longer-term shift in demand, and they're busy retooling to meet it.

The real question is whether NVIDIA will maintain its lock on the market, or if vendor-agnostic Torch will help commoditize the segment.

[arvinsim]

I am also thinking if I should build a rig to bet against expensive GPU prices.

My fear is that Nvidia trying to anchor GPU prices to crypto levels whether there is TSMC shortage is not.

[Tostino]

"thanks Facebook" was not a term I have ever actually wanted to utter.

[drBonkers]

> vendor-agnostic Torch

Is this a competitor to CUDA?

[Roark66]

Cuda runs on a lower layer than torch. There is not much competition to cuda if one wants performance, but if one just wants to run it there is cpu (openblas etc and Intel's MKL) and onnx.

It's pretty astonishing to me AMD has neither a proper math library (like MKL) nor gpu compute library (like cuda).