[dharmab]

Efficiency is an area where there can be a lot of cost hiding. We recently saved a lot of money by:

- Using the Kubernetes Vertical Pod Autoscaler (https://github.com/kubernetes/autoscaler/tree/master/vertica...) for CPU and memory scaling, and switching to metrics like requests per second and connection count for horizontal scaling

- Collecting metrics on container CPU and memory allocation vs utilization

- Writing scripts to auto-submit PRs to SREs with better recommended sizing based on actual usage

- Tuning our VM instance sizes and autoscaler configs

A few engineers were able to save the company several times their salary with a few months of work, and plan to 10x the savings over the next year

[najarvg]

Would love to see a more detailed write-up, if one exists, on your experience. We're nowhere near your scale but will probably ramp up in the next year or so.

[avip]

Pre-scale just overprovision and pay $ to Bezos instead of $$ your precious engineers. 96% you won't "ramp up" anyhow.

[colechristensen]

During a ramp up the limiting factor is usually engineer hours, new features, and ability to have the load at all and not cost.

[dharmab]

I'm going to try to convince the lead on this project to present at an upcoming Kubecon :)

[qubit23]

If you ever open-source your rightsizing scripts, I think a lot of people (including myself) would find that useful. While the vertical pod autoscaler attempts to solve this problem, it's incompatible with horizontal pod autoscalers based on CPU and memory, and thus unusable in a lot of cases. Going with a semi-automated solution might be ideal.

[dharmab]

You can just run vpa-recommender without vpa-updater to get VPA recommendations without the autoscaling. That should work for most people.

[justicezyx]

Very nice! I suspect the economy of Cloud will soon change in such a way that Cloud providers will be focusing more on Enterprise, which have very little capability of doing similar things. And then, a lot of companies like Hakuna Cloud (reported here) will emerge, just like what happened in the last iteration of IT infrastructure.

[totomz]

Actually, HakunaCloud works on cloud servers, not containers. With Kubernetes you pay for the worker nodes, not for running containers.

HakuaCloud starto&stop cloud servers, so you actually pay only for the time the servers are needed

[dharmab]

The number of running nodes is a direct function of the number and size of your containers. The cluster autoscaler stops and and start servers based on how many containers are desired, which is in turn based on the Horizontal Pod Autoscaler inputs (e.g. incoming requests).

[cbluth]

> Tuning our VM instance sizes and autoscaler configs

Do you run kubernetes in VMs on baremetal?

[dharmab]

Yes, we own some datacenters as well as use multiple cloud providers. You are still responsible for running your own worker VMs with a managed Kubernetes anyway.

[ma2rten]

I am confused by your use of the word cost hiding. Do you mean cost saving?

[dharmab]

as in hidden costs- costs that the organization is unaware of because it's wrapped up in r&d/goods delivered on the accounting sheets. Its='s not until you pull in the utilization metrics that it reveals itself as waste.