[meigetsu]

Since the GPU uses the system memory, is there any advantage to using these APUs for machine learning?

GPU RAM is typically under 32GB (more commonly under 11GB) and quite expensive - for the price of a V100 you could buy 1TB of system DDR4 RAM.

I'm guessing there are disadvantages in memory bandwidth, number of GPU cores, overall FLOPS, but was curious if anyone knew how these pros/cons balance out.

[nickysielicki]

The big issue I'd expect you to hit is that you're doing ML without CUDA, and there's just inherent problems to working in an environment where your hardware isn't the first-class citizen.

ROCm/HIP/HCC is a broken ecosystem as of today. The 5700XT launched last year and still doesn't have support. I have some faith that they're going to get it straightened out in the next year or so, but today you're just asking for trouble if you're trying to use pytorch or tensorflow without being on team green.

[pmalynin]

I was an ML engineer in a previous life.

For most ML and GPGPU workloads the bottleneck is memory bandwidth. It’s normal to hit 2TB/s because of all the caches, memory banks and the wide memory bus. I wouldn’t expect the APU to perform nearly as well as even an entry level GPU (say 1660) but I would be curious to see.

[justincormack]

These machines are limited to 64GB RAM AFAIK, or maybe 32GB.

[AdrianB1]

The memory controller should support 4 DIMMs of 32 GB each, but in practice the motherboard implementation may limit to 4 x 16GB. How much can be allocated to the GPU part, no idea.