[klft]

> Using Microsoft Olive and DirectML instead of the PyTorch pathway results in the AMD 7900 XTX going form a measly 1.87 iterations per second to 18.59 iterations per second!

So the headline should be Microsoft Olive vs. PyTorch and not AMD vs. Nvidia.

[mananaysiempre]

The results of the usual benchmarks are inconclusive between the 7900 XTX and the 4080, Nvidia is only somewhat more expensive, yet CUDA is much more popular than anything AMD is allowed to support. So I’d say this makes sense as an AMD vs Nvidia comparison as well.

[dangus]

The existence of the 4090 is another issue.

I’m not sure which customer willing to spend $1000-1200 to do ML workflows isn’t willing to spend $1600 to get another 20%+ of performance and have the fastest card available.

I’m not saying people have unlimited budgets but it just seems like the choice most people in that price range would make.

[selfhoster11]

While hobbyist users are not a significant chunk of the market, you can almost bet that they will want the best bang for buck irrespective of the performance of an individual card.

Tesla P40 is being heavily discussed in these circles because although it's janky (extremely bad TFLOPS for 16-bit operations), it's still thought to be a decent option for inference due to the high amount of VRAM at its price point.

[3abiton]

Any references on those P40 discussions?

[Aerroon]

This raises potential for the next AMD generation though. If they can reach better cost effectiveness but provide more VRAM then that could work.

[ineedasername]

If it’s completely down to Olive and DirectML then nvidia should be able to use them for similar performance improvements. If not, then AMD is still a defining factor. A quick search didn’t bring up anything definitive on the question though, so I guess we’ll have to wait for someone to try it out ( or someone with faster Google-fu than me)