[rvnx]

At some point Google Cloud, AWS, Alibaba Cloud, Apple, etc are going to make their own specialized chips (Google tried a bit with their tensors chip).

There is no value into the NVIDIA-part by itself, only the raw power is interesting.

If tomorrow this is AMD, or China-Town chip, it's perfectly fine.

I wouldn't miss the CUDA toolkit mess.

[KeplerBoy]

If raw power per dollar would be all that's interesting we'd all run 7900 XTX clusters like geohot in his tinybox.

We are not, because there's clearly value in the CUDA ecosystem.

[latchkey]

There certainly is a lot of value in the CUDA ecosystem, today. The problem is that when all the big companies are buying up hundreds of thousands of GPUs, that doesn't leave much for anyone else.

Sane business people will look to decentralize their compute over time and not be reliant on a single provider. AMD will be able to take advantage of that and they've already stated that is their focus going forward.

ROCm/HIP are getting incrementally better, MI300x have 192GB and benchmarks are looking good, the only problem is that nobody has access to the higher end hardware from AMD today. That's why I'll have MI300x, for rent, soon.

[KeplerBoy]

That's a big issue in AMD land imho. Everyone can pickup a 200$ GPU (talking about the RTX 3050) which will behave like a scaled down A100 and get started playing around with CUDA. You can't really do that with AMD GPUs, their cheapest officially supported GPU is the 7900 XTX and that has a different architecture than the data center ones.

[latchkey]

I agree. Maybe one idea would be to also make 7900 XTX's for rent (cheaply) too.

[KeplerBoy]

That's another thing. I have some stuff I'd like to try, but I can't even find places where I could quickly rent a GPU without applying for quotas.

[latchkey]

That is indeed an issue, and I am actively working on it.