[adrian_b]

You can go past this only on a matmul benchmark, which is seldom useful per se.

Linpack consists mostly of matmuls, but nonetheless there are additional operations that prevent GPUs to reach the high utilization of over 80% that is normal for CPUs, so that a throughput over 50% is considered good at the scale of supercomputers.

At the scale of a supercomputer, the utilization factor is considerably less than for an individual GPU or CPU, because the big matrix is split in blocks and the matrix multiplications are computed on different boards and in different racks, then the results are assembled, so there is a communication overhead.

The former Intel Xeon Phi, with a large number of cores that were weak except for their vector execution units, resembled GPUs in failing to reach a high utilization on Linpack.

[saagarjha]

Yeah, I mean, if your workload is oddly shaped it will definitely work better on CPU than GPU. CPUs are easier to get higher utilization on but I feel like they are almost cheating, because maxing out a core is really translating your code to a machine that has weaker performance (die area, etc.) in exchange for it handling some of the complexities of your algorithm.