[rumdonut]

It wouldn’t be hard to stop. Chip fabrication is a concentrated industry with a high barrier to entry (indeed there are only a few companies in the world producing high performance silicon using technology made by probably even fewer companies). Restrict AI chip making and the materials needed, and you’ll restrict AI. I can see global treaties between large nations effectively controlling production in the same way that we have nuclear treaties that prevent proliferation or enrichment.

[nuancebydefault]

AI chip making? I can train an AI on my intel laptop if I whish. If I need more CPU power, i can rent some. The genie is out of the bottle and the only way is forward. The latest worldwide race.

[avital]

This isn't accurate. The bottleneck in very-large-scale-training BY FAR is communication between devices. If you have a million CPUs, the communication cost will be significantly higher than a thousand A100s (perhaps in the order of 100x or even more). So this is only possible to replicate with very dense and high compute chips with extremely fast interconnect.

[nuancebydefault]

Thanks for providing this insight. Is A100 the only platform? Can we pause/resume all such platforms simultaneously?