| While some years might pass until they will really catch up, that does not prevent them to find workarounds for their weaknesses. For example, they recently have demonstrated a supercomputer faster than any of the US supercomputers. Unlike the recent European supercomputers, which like the US supercomputers have been built by buying racks from HPE Cray, because China was not allowed to buy such things they have developed their own custom CPUs, designed in China, which have surpassed in throughput the AMD GPUs used in the fastest US supercomputers. The Chinese CPUs match in memory bandwidth per socket the latest AMD MI355X GPUs (8 TB/s), while being significantly faster than the older AMD GPUs installed in the US supercomputers. While the purpose of the new Chinese supercomputer is mainly for scientific/technical computing tasks that need high FP64 throughput, the CPUs used in it also have high enough BF16/INT8 performance and memory bandwidth and interconnection bandwidth (1.6 Tb/s directly from each CPU socket) to be able to train any big LLM. So the evidence does not show China falling behind, but at least in certain directions they are already exceeding the performance of what they have been forbidden to buy. For something like training a big LLM, the only disadvantage of the current Chinese devices is a lower energy efficiency, of only 65% to 70% of that of the best NVIDIA GPUs. However that is not really a problem for China, as they have abundant cheap energy. |