[cbushko]

HPAs can definitely save you a lot of money when running Kubernetes and they are extremely useful, especially for non-production environments where you want to be efficient as possible.

Strategies I have used in the past for saving money are:

  1) Set requests very low for your pods. Look at the minimum CPU/Memory that your pods need go start and set it to that. Limits can be whatever.

  2) Set min replicas to 1. This is a non-production environment, nobody cares if an idle pod goes away in the middle of the night.

  3) Use spot instances for your cluster nodes. 80% savings is nice!

  4) Increase the number of allowed pods per node. GKE sets the default to 110 pods per node but it can be increased.

  5) Evaluate your nodes and determine if it makes more sense to have `fewer large sized nodes` or `several smaller nodes`. If you have a lot of daemonsets then maybe it makes sense to have fewer large nodes.

  6) Look at the CPU and Memory utilization of your nodes. Are you using a lot of CPU but not much memory?  Maybe you need to change the machine type you are using so that you get close(r) to 100% CPU and Memory utilization. You are just wasting money if you are only using 50% of the available memory of your nodes.

  7) Use something like knative or KEDA for 'intelligent autoscaling'. I've used both extensively and I found KEDA to be considerably simpler to use. Being able to scale services down to 0 pods is extremely nice!

[WarChortle]

> Set requests very low for your pods. Look at the minimum CPU/Memory that your pods need go start and set it to that. Limits can be whatever.

Wouldn't this lead to Node Over provisioning?

I ask because my company's workload is very spiky and usage is very minimal until it isn't. We are looking into ways to optimize it.

[cbushko]

My example was for non-prod and saving money there as I found that our development clusters tended to be the most under utilized per dollar spent. In development it was ok to put as many idle pods as possible on the nodes. If there was a spike, then yes you could get new nodes but I found that they scaled down nodes quite often.

My apologies in advance as the advice can be terrible depending on your environment and services. Below is not an exact science as you are dealing with requests and limits while trying to find optimal performance.

For production you need to calculate your minimum, average and max CPU/Memory for your a pod.

  1) Set your replicas to 1

  2) Determine what your true maximum CPU/Memory is for a pod. 

  Set your limits to very high and performance test against your pod. If your response time slows to a crawl then your limit is too high and your code may not be able to handle the load. If your response time is good while hitting the limit, increase the limit until performance goes down.

  3) Get your minimum CPU/Memory for your pod to start.

  5) Get your average CPU/Memory DURING THE SPIKES. You should be able to get this from past metrics. This can also be difficult to get because your load might be spread over several pods in your metrics.

  6) I use the following formulas:

     requests = (min + average)/2
     limits = (average + max)/2

  7) You now have a baseline for the future so that you can tweak the values.

  8) Set your autoscaler to something high like 80% CPU. You want this value to stay constant. I think GKE sets it to 60% but I found that to be far too low and wasteful.

  9) Observe and tweak the values to see if you can get things 'better' depending on your needs.

There are two other things I always do in production that help with stability and reliability.

  - Set the autoscaler behaviour to scale up quickly and scale down slowly. It stops these cycles of add 3 pods, remove 1 pod, add 2 pods, remove 3 pods chaos in short periods of time during spikes. The behavior field was added to the autoscaler resource a couple releases ago.

  - Set your minimum replicas to 2 for redundancy. I always do this in production.

I hope this helps and I apologize once again for the hand wavyness of things.