Related question, does anyone have experience with using the AMD MI100 for deep learning? With 32GB and a second hand price of ~1100 USD, it could be a good choice.
I've been really curious about these, but my experience with an MI60 and partially my 6900XT has not endeared me towards using AMD cards - the MI60 refuses to init in linux, due to some PSP firmware issue, and the 6900xt is missing pre-compiled HIP stuff leading to super long initial launches - as. it JIT builds the kernels - at least in PyTorch.
Allegedly they perform near an A100, so raw-compute wise, memory capacity wise, and memory bandwidth wise, they rock. As is typical for anyone not Nvidia, the software is still playing catch up. To be fair, Nvidia themselves takes nearly a year to build out all CUDA features for some of their cards - FP8 for example, is only recently become usable on a 4090.
Allegedly they perform near an A100, so raw-compute wise, memory capacity wise, and memory bandwidth wise, they rock. As is typical for anyone not Nvidia, the software is still playing catch up. To be fair, Nvidia themselves takes nearly a year to build out all CUDA features for some of their cards - FP8 for example, is only recently become usable on a 4090.