[adrian_b]

If you can avoid GPUs, that is preferable.

The reason is that with GPUs it is far more difficult to reach a great percentage of the maximum theoretical throughput. Most GPU programs reach only a very small fraction of what is theoretically possible, and in the best cases one may reach something like 50% to 60% of the maximum.

This CPU-based supercomputer has demonstrated reaching 80% of the theoretical maximum throughput, and this is typical for CPU-based supercomputers. It is much easier to write efficient programs for CPUs.

The new custom Chinese CPUs, which use SME, the Arm Scalable Matrix Extension, are fast enough that they have beaten all GPU-based supercomputers, so there was no need to use GPUs.

Moreover these CPUs use HBM for a very fast memory interface, so in the benchmarks that depend more on memory bandwidth they have an even greater advance over the US GPU-based supercomputers. Thus there really was no point in using GPUs.

GPUs are necessary only when your CPUs are not good enough, which was not the case here.

In the recent past, the Japanese Fugaku used the same approach, of avoiding GPUs. At that time, their custom CPUs using the Armv8-A ISA with SVE were the first which used this ISA in HPC, but now that ISA variant is obsolete in comparison with the Armv9-A ISA with SME, which is implemented in these new custom Chinese CPUs.

[saagarjha]

> in the best cases one may reach something like 50% to 60% of the maximum

If all you need to do is matmuls then you can definitely go past this

[adrian_b]

You can go past this only on a matmul benchmark, which is seldom useful per se.

Linpack consists mostly of matmuls, but nonetheless there are additional operations that prevent GPUs to reach the high utilization of over 80% that is normal for CPUs, so that a throughput over 50% is considered good at the scale of supercomputers.

At the scale of a supercomputer, the utilization factor is considerably less than for an individual GPU or CPU, because the big matrix is split in blocks and the matrix multiplications are computed on different boards and in different racks, then the results are assembled, so there is a communication overhead.

The former Intel Xeon Phi, with a large number of cores that were weak except for their vector execution units, resembled GPUs in failing to reach a high utilization on Linpack.

[saagarjha]

Yeah, I mean, if your workload is oddly shaped it will definitely work better on CPU than GPU. CPUs are easier to get higher utilization on but I feel like they are almost cheating, because maxing out a core is really translating your code to a machine that has weaker performance (die area, etc.) in exchange for it handling some of the complexities of your algorithm.