$10m per training run gets me a lot of engineering time to build our own version of this system and lease it to other customers. Just skip one training run and I've got a pretty good team.
Putting aside the question of whether it would ever be a choice between spending $10M on a training run and hiring a team for $10M, GPT transformers were the end result of decades of language research and innovations. You’re making it sound as though you can build the next iteration past GPT-3 for $10M, which I don’t think is the case.