[adrian_b]

"Useless" means that both DP Gflops/s/W and DP Gflops/s/$ are worse for the modern AMD and NVIDIA gaming GPUs, than for many CPUs, so the latter are a better choice for such computations.

The opposite relationship between many AMD GPUs and the available CPUs was true until 5-6 years ago, while NVIDIA had reduced the DP computation abilities of their non-datacenter GPUs many years before AMD, despite their previous aggressive claims about GPGPU being the future of computation, which eventually proved to be true only for companies and governments with exceedingly deep pockets.

[Const-me]

My desktop PC has Ryzen 7 5700G, on paper it can do 486 GFlops FP64 (8 cores at 3.8 GHz base frequency, two 4-wide FMAs every cycle). However, that would require 2TB/sec memory bandwidth, while the actual figure is 51 GB/second of that bandwidth. For large computational tasks where the source data doesn’t fit in caches, the CPU can only achieve a small fraction of the theoretical peak performance ‘coz bottlenecked by memory.

The memory in graphics cards is an order of magnitude faster, my current one has 480 GB/sec of that bandwidth. For this reason, even gaming GPUs can be much faster than CPUs on some workloads, despite the theoretical peak FP64 GFlops number is about the same.

[adrian_b]

You are right that there are problems whose solving speed is limited by the memory bandwidth, and for such problems GPUs may be better than CPUs.

Nevertheless, many of the problems of this kind require more memory than the 8 GB or 16 GB that are available on cheap GPUs, so the CPUs remain better for those.

On the other hand, there are a lot of problems whose time-consuming part can be reduced to multiplications of dense matrices. During the solution of all such problems, the CPUs will reach a large fraction of their maximum computational speed, regardless whether the operands fit in the caches or not (when they do not fit, the operations can be decomposed into sub-operations on cache-sized blocks, and in such algorithms the cache lines are reused enough times so that the time used for transfers does not matter).

[Const-me]

I guess I was lucky with the CAM/CAE software I’m working on. We don’t have too many GB of data, the stuff fits in VRAM of inexpensive consumer cards.

One typical problem is multiplying dense vector by a sparse matrix. Unlike multiplication of two dense matrices, I don’t think it’s possible to decompose into manageable pieces which would fit into caches to saturate the FP64 math of the CPU cores.

We have tested our software on nVidia Teslas in a cloud (the expensive ones with many theoretical TFlops of FP64 compute), the performance wasn’t too impressive.