|
|
|
|
|
|
by adabyron
718 days ago
|
|
|
I have read in a few places that Microsoft is using AMD for inference to run ChatGPT. If I recall they said the price/performance was better. I'm curious if that's just because they can't get enough Nvidia GPUs or if the price/performance is actually that much better. |
|
|
Think of it this way: AMD is pretty good at hardware, so there's no reason to think that the raw difference in terms of flops is significant in either direction. It may go in AMD's favor sometimes and Nvidia's other times.
What AMD traditionally couldn't do was software, so those AMD GPUs are sold at a discount (compared to Nvidia), giving you better price/performance if you can use them.
Surely Microsoft is operating GPUs at large enough scale that they can pay a few people to paper over the software deficiencies so that they can use the AMD GPUs and still end up ahead in terms of overall price/performance.