[msarnowicz]

Hey, author here. Please AMA.

I came into the Linux world via Postgres, and this was an interesting project for me learning more about Linux internals. While cgroups v2 do offer basic support for CPU bursting, the bursts are short-lived, and credits don’t persist beyond sub-second intervals. If you’ve run into scenarios where more adaptive or sustained bursting would help, we’d love to hear about them. Knowing your use cases will help shape what we build next.

[motrm]

Echoing parrit's comment, this was indeed a very nice read and very well written.

I particularly enjoyed the gentle exposition into the world of cgroups and how they work, the levers available, and finally how Ubicloud uses them.

Looking forward to reading how you handle burst credits over longer periods, once you implement that :)

Lovely work, Maciek!

[msarnowicz]

Thank you very much, I appreciate your comment.

[nighthawk454]

Great article, thanks! I’ve been curious if there’s any scheduling optimizations for workloads that are extremely burst-y. Such as super low traffic websites or cron job type work - where you may want your database ‘provisioned’ all the time, but realistically it won’t get anywhere near even the 50% cpu minimum at any kind of sustained rate. Presumably those could be hosted at even a fraction of the burst cost. Is that a use case Ubicloud has considered?

[msarnowicz]

This is a very valid scenario, however, one that is not yet fully baked into this implementation. But, as mentioned, this is a starting point. We want to hear feedback and see customers' workloads on Burstables first.

The main challenge here is that cpu.max.burst can be set no higher than the limit set in cpu.max. This limits our options to some extent. But we can still look at some possible implementation choices here: - Pack more VMs into the same slice/group, and with that lower the minimum CPU guaranteed, and at the same time lower the price point. This would increase the chance of running into a "noisy neighbor", but we expect it would not be used for any critical workload. - Implement calculation of CPU credits outside of the kernel and change the CPU max and burst limits dynamically over an extended period of time (hours and days, instead of sub-second).

[nighthawk454]

Gotcha, thanks for the reply. Makes sense to target burstables first - that seems to be the most common feature set. That’s interesting that it’s not readily available in the kernel. I once spoke to some AWS folks dealing with Batch/ECS scheduling of docker container tasks and they hit similar limitations. As a result their CPU max/burst settings work like the underlying cgroups too.

I imagine writing a custom scheduler would be quite an undertaking!

[msarnowicz]

I think so, too!

[parrit]

Thanks! That was a pleasant read. I have been wanting to mess with cgroups for a while, in order to hack together a "docker" like many have done before to understand it better. This will help!

Are there typical use cases where you reach for cgroups directly instead of using the container abstraction?

[msarnowicz]

Thanks for the kind words. Even if you are not building a cloud service, I think it is good to understand how the underlying layer works and what are the knobs and the limits of the platform. I could see a use case where two or more processes need to run on one VM or a container, maybe for cost-saving reasons or specific architecture/security reasons, but need to be guaranteed a certain amount of resources and a certain isolation from each other.

[phrotoma]

I don't have a question but I really wanted to say thanks for the blog post. Extremely clear and cogent writing on a tricky topic. Well done!