[tekkk]

I hope that the original title of the story was intended sarcasm: "Unbabel migrated to Kubernetes and you won’t believe what happened next!"

But so they managed to consolidate their infrastructure around Kubernetes and Google Cloud which made the management of their servers easier and faster? I wonder how much actual money they saved but I guess it will pay off for them in the long run.

I've been dabbling with Kubernetes for some time now but God forbid it can be a bit complicated. Time required to become well-versed with Kubernetes is a hefty investment which is not for all organizations. Lots of small things that can drive up your blood-pressure when figuring them out. Were it simpler I would be much more inclined to be using it but now it's only in the "learning for funsies" -category. I feel people who've developed k8s have been more of the theoretical sort and not the regular-joe-dummy-kind like me.

[falcolas]

Saying that Kubernetes is a bit complicated seems like saying that water can be a bit wet.

Even their documentation can't keep up. And with a release cycle of 3 months, and a deprecation cycle of 6 months, you need a team dedicated to keeping up with K8s state-of-the-art; so much of that knowledge you picked up a year ago is at best stale, and at worst wrong.

Sure, it makes setting up and keeping a set of containers up simple. But that's never really been that hard.

To paraphrase an article from a few weeks ago:

"We made microservices to address the problems with monoliths."

"We made containers to address the problems with microservices."

"We made Kubernetes to address the problems with containers."

[eeZah7Ux]

"Now we have a distributed monolith that requires 2x less developers to build and 5x more system engineers to deploy"

[falcolas]

Well, to be fair, all of those developers probably found themselves filling a systems engineer role "because the product developers are best equipped to handle the running and support of their own applications".

[cdelsolar]

deploying: `kubectl apply -f file_with_changed_docker_image.yaml`

[halbritt]

My org made a similar transition. It's hard to articulate exactly precisely how much we saved as our "production" or revenue generating environments are roughly the same.

We transitioned out of AWS where we had relatively well managed instances of our stack managed with chef and terraform to GKE in Google Cloud where we migrated to a helm chart and custom orchestration tooling on top of that.

Prior to the migration I'd say that 80% of our instances were idle. Currently, all of our k8s nodes with 16 cores are running with an average load 5-7. We try to keep enough headroom to prevent any waiting or queuing, which is an entire medium blog post unto itself.

So, roughly the same or a little more "production" workload, but the number of non-prod instances of our stack quadrupled. Anyone in the org at any time can spin up an instance of the stack for a custom sales demo, to debug an issue, to test a feature, or anything else. There was a great deal of pent up demand that nobody expected and which caused my team to thrash a bit to catch up to when we made the transition.

All in all, our GCP spend is about 20% less than our AWS spend was. We're getting a lot more utility for a little less money.

[falcolas]

Idle instances are what autoscaling groups were made for. Frankly, if there are a lot of idle instances, AWS features are not being taken advantage of - that's not on them.

[adwf]

Only if you've got a homogeneous workload that can scale out.

More typical is Team A working on Project A spin up a test DB + test app server. Team B also spin up a test app server + web server + DB for project B

Then Project A gets productionized and you have SIT/UAT/Stage copies of all of that sitting mostly idle.

Then project C comes along and the devs need to test on a 3 node C* cluster with 3 kafka brokers...

Suddenly you have a whole bunch of dev environments sitting mostly idle. No-one would ever fork out on reserving a t2.medium, but they all add up to $$$$$ every month. With k8s you can reclaim all that idling power, reserve some beefy instances, whilst also gaining easily deployable artefacts, CI/CD, production scaling, etc.

[falcolas]

But those use cases don't change between AWS and GCE. They only change if you start using containers in a container environment; something possible in AWS and GCE.

[halbritt]

The distinction is between traditional config management and instances vs. Kubernetes. I could've seen similar benefits running k8s in AWS. I simply chose GKE because it's superior to other offerings.

There are other things to like about GCP vs. AWS, but that's a bit tangential.

[wgjordan]

Seconded. If the primary driver of this transition was 'getting a lot more utility for a little less money', then simply adopting Auto Scaling Groups within the existing AWS setup could have improved instance-utilization with much less migration effort.