At Canva I built auto-scaling GPU infra on K8s for model training[1], and it's way too much work and operational expense to be worth building yourself. I went work at Modal because building it properly once and then distributing the solution was going to be just way better and more efficient.
I'm not sure why you are getting downvoted, probably because people feel you are advertising Modal.
But I have to say something about Modal. The difference with this vendor is that they try to reimagine the way people build on the Cloud and it's worth checking out just to see how different the developer experience could be.
I know that most people use it because of the easy and affordable access to GPUs, but I think we are missing the true innovation here, which is the developer experience.
I would even consider Modal as a cloud infra product, although a vertical one, more than an ML or DE product.
Didn't realize it was downvoted, but fair enough if people feel it's too much of an ad. Comment is sitting at 2 points now :)
Glad you really get what we're trying to do with Modal. You're right it's not just an easy way to get serverless GPUs.
Modal is reimagining software development practices for the cloud era. Developing in the cloud should not be just writing YAML or Hashicorp Config Language templates, push/pulling Docker images, and re-running 'infratool up' over and over until things over.
I talk to people who want to set up infra to use cloud GPUs and many of them say "I want to use Modal".
Common reasons not to include (1) "I have soooo many AWS credits that I want to use" and (2) (our company's reason) "We have on-prem GPUs but sometimes need Cloud GPUs as well with the same interface".
Using e.g. Ray with AWS is very painful, took us a long time to iron out all the quirks.
But I have to say something about Modal. The difference with this vendor is that they try to reimagine the way people build on the Cloud and it's worth checking out just to see how different the developer experience could be.
I know that most people use it because of the easy and affordable access to GPUs, but I think we are missing the true innovation here, which is the developer experience.
I would even consider Modal as a cloud infra product, although a vertical one, more than an ML or DE product.
*edited to fix some spelling*