I'm not referring to overcommit or actual "cpus pegged at 100%" contention, but to simple loose bin-packing.
Imagine you have three Ruby services, where each is allocated 10 cores of CPU time (via pinning with cpuset). If you give them each an 16-core VM, then there'll be 18 cores of "wasted" CPU. If you instead bin-pack them onto a 32-core VM, then they'll have the same number of cores at a lower price point.
If each service runs at 50% capacity with 2000ms latency during steady state, how much extra latency would you expect the service to have on the bin-packed configuration vs the single-VM?
My position is "very little extra latency", the other person's position was "a lot of extra latency due to hardware contention in (for example) the memory controller".
(If you're reading this and thinking "NUMA node locality", then you're operating two or three levels above where this org was in terms of optimization.)
> If you're reading this and thinking "NUMA node locality", then you're operating two or three levels above where this org was in terms of optimization.
Talking about Ruby services and not hand optimized C kinds of give it away. And even with hand optimized C you would do a cost/benefit analysis of less optimal packing.
Imagine you have three Ruby services, where each is allocated 10 cores of CPU time (via pinning with cpuset). If you give them each an 16-core VM, then there'll be 18 cores of "wasted" CPU. If you instead bin-pack them onto a 32-core VM, then they'll have the same number of cores at a lower price point.
If each service runs at 50% capacity with 2000ms latency during steady state, how much extra latency would you expect the service to have on the bin-packed configuration vs the single-VM?
My position is "very little extra latency", the other person's position was "a lot of extra latency due to hardware contention in (for example) the memory controller".
(If you're reading this and thinking "NUMA node locality", then you're operating two or three levels above where this org was in terms of optimization.)