[gorbypark]

The problem would be that orgs like Meta would stop publishing llama 3/4/5/etc, which most open source models build upon. Without new foundational models, progress would stall pretty quickly, and procuring thousands of GPUs to train new foundational models would be difficult. In theory, since the US “controls” Nvidia/amd/tsmc, they could put up roadblocks to even doing open training outside of the US. Maybe a “SETI@Home” style distributed training system could be done on consumer GPUs…

[Chris2048]

> since the US “controls” Nvidia/amd/tsmc

They don't control China or Europe, and will hand their overseas monopoly to overseas competition.

[gorbypark]

China is the only place that conceivably in the near future would be able to spin up their own designs and fabs for GPUs. In fact they seem to be well on their way, thanks to the push from the GPU export restrictions already in place!

[Manabu-eo]

Training needs lots of bandwidth, at least as done today. Something easier to distribute and do in a small scale is dataset creation.