[dekhn]

Pretty much everybody who trains models trains tens to hundreds of models before their final run. But a better way to think about this is: Google spent money to develop the TPUv4, then paid money to have a bunch built, and they're sitting in data centers, either consuming electricity or not. Google has clearly decided the value of these results exceeds the amortized cost of running the TPU fleet (which is an enormous expense).