[xenospn]

Read up on the datacenter OpenAI had to spin-up within Azure in order to train GPT3. A little out of reach for now - but in 10 years, who knows.

[jchw]

To be fair, training is a lot more intensive than inference. Though if open source models are any indication, the big issue is actually VRAM requirements.