[thisisjeffchen]

Yeah the main use case here is training, which is much more of an offline process. For inference, you will probably want a cloud provider. Though the cost difference still holds, so maybe a startup idea.

[mr_toad]

Inference can usually be done on much cheaper hardware, or using lambda.