[accrual]

Agree. It's a terrible time to want a new Nvidia card for gaming or AI projects. I had a 4070 Ti Super in my cart for a while at a cool $700 on Amazon - now it's out of stock and secondhand I can find it for double that now.

If one doesn't require the latest DLSS or need first-class AI support, get an AMD card.

[SV_BubbleTime]

AMD’s is coming up in support for things like torch and xformers. But it isn’t there yet.

If you want for AI, your best bet is a used 3090 for $700-800. The ram is more important and that card is still faster than a 4070.

[codedokode]

AMD has documentation for its assembly language as I remember. Or one needs more documentation to run AI?

[SV_BubbleTime]

Isn’t a documentation issue.

The entire ecosystem is built around CUDA currently. There was someone implementing Cuda on AMD hardware but as I understand it, AMD shut that down.

There are two alternatives to CUDA for AMD right now, but I forget their names.

AMD has STRIX coming, which is a single board embedded ram machine, and is being targeted at AI loads with lots of low latency ram. So we’ll see what happens there.

[colonCapitalDee]

Thankfully the ONNX runtime supports AMD's ROCm. The performance is nowhere close to Nvidia's TensorRT, but doing inference on an AMD GPU is doable. I haven't ran the numbers, but I wouldn't be surprised if you could get more throughput per dollar on an AMD GPU for at least some scenarios. AFAIK there aren't any great options for training though, so if you want to do anything besides running ONNX models your options are limited.