[MichaelRazum]

Grok4 was trained on 100k or 200k GPUs (as far as I understand)

Grok5 might need 1MM or 2MM.

So the question is what about metas / zucks plans? How many GPUs will Manhattan get? Looks like, that to get the next unlock you need crazy amounts of compute.

[jiggawatts]

Meta had the equivalent of about 600K H100 cards a year ago, but they were geographically distributed and used mostly for inference.

These giant data centres will allow these companies to put about a million in one location and possibly into a single giant training cluster.