[halfeatenpie]

You can rent cluster access (to an extent) using Azure Batch.

Granted it's probably not at this scale, but it gives you access to a ton of resources.

[ankeshanand]

You can also rent a cloud TPU-v4 pod (https://cloud.google.com/tpu) which 4096 TPUv-4 chips with fast interconnect, amounting to around 1.1 exaflops of compute. It won't be cheap though (excess of 20M$/year I believe).