[shortrounddev2]

I think if you had a whole server rack, you might have to hire someone for that, but if you're just running one or two servers with containers to run your dev builds, you could get that set up on K8s and manage your deployments the way you would with AWS. I don't think it would require hiring new people, an existing devops guy should be able to do that

[alephnerd]

It's still additional operational overhead for marginal RoI.

If you're an organization that already has a SRE or Infra Eng hired, there's no point supporting a niche setup when every other team is using a prebuilt and troubleshooted environment.

Edit: Can't reply to OP atm, but he makes a good point. For a higher margins and less commodified segment like ML Compute, on-prem computing still makes financial sense.

That said, that's still managed by actual SWEs not a generic Sysadmin, due to intricacies of GPU Architecture and Model Development, and OP is extremely underskilled for this role.

[shortrounddev2]

I think if all you're running is web microservices, then it's probably not worth it. But my company builds machine learning models offline and I feel that if we bought a dedicated GPU machine (literally just build one and put it in the office, not a rack mounted thing), we would save money in the long run after building a custom training pipeline