Well yes, but I run container orchestration on owned servers (Swarm, not Kubernetes) so it's really easy to add a new service at a whim and only quite rarely do we need to add capacity, to which we can also add in a cloud provider. But I'm using off-lease servers which are sort of bonkers as to how much capacity you get for the price. I think they cost less than a months spend on an equivalent cloud instance.
That’s a good point. Preemptable instance can be insanely cheap. I would guess it’s more efficient for the data center overall to shuttle services around to different nodes as needed.