[chias]

I was in this state a few weeks ago. I spent a bit of time familiarizing myself then wrote up my learnings as a series of exercises.

If you think of docker as "kinda like vms except not really" and k8s as "kinda like deploying and composing docker containers but not really", this may be for you:

https://ojensen.net/infra/understanding-k8s-1

It's actually really neat, i wish i had bothered to learn it years ago.

[munchler]

I appreciate the effort and I'm in your target audience, but that document didn't help me. It seems to dive into the details of installing and running k8s without saying much about the purpose.

From my very ignorant standpoint, K8s seems to be about running a "cluster", but I don't know why I would want to do that.

[what-is-water]

Kubernetes orchestrates your container workloads over a cluster, which consists of virtual/bare metal machines(nodes). This means you can tell the kubernetes API "I want to run a container workload" and it will be started on one of the nodes that form the cluster, unlike e.g. Docker, where a docker daemon belongs to a specific node. If you remove the node your workload is running on from the cluster the workload will be rescheduled on a different one, or if you have a new image version it will start the new container, wait for it to become healthy and ready, and then route requests to it. And if you want to send requests to your workload kubernetes allows you to define standardized abstractions to easily route them to your workload, irrespective of the node it is running on.

It allows you to stop caring about the individual machines, and just treat them as combined compute, which starts mattering if you leave a single machine setup and need to start thinking about scaling in and out and gluing the individual parts together. Then you have known abstractions to do it.

Of course you can do everything kubernetes does using a bespoke solution, and the concepts aren't new, but having a widely supported technology has a lot of advantages and creating something with even half the feature has a high chance of just being worse.

[munchler]

Thank you for this explanation. It makes sense, but I don't really understand why it has become so popular.

Professionally, my experience is that certain software components need to run together on an individual machine (e.g. database server, app server, web server), and then those machines need to be networked in a certain way (e.g. web server talks to app server, which talks to database server), so I really need to care about the architecture of individual machines. You can then scale this out horizontally (e.g. add another web server) or vertically (e.g. upgrade your database server).

I'm old, so maybe I'm out of date, but having a cluster of "compute" that I can run arbitrary workloads on sounds neat, but is a capability that I've never needed.

[Krei-se]

Your host with the webserver has no idea its just a docker image. It talks to app.domain.tld and gets a reply. app.domain.tld asks db.domain.tld with an SQL query and gets a reply. But it can be one of any deployed docker image on any bare metal host - which one is db. and app. etc. is decided by Kubernetes.

In the background kubernetes routes all these docker images with each other without you having to think about this. You may have 1000 db.domain.tld nodes caching a master db host - if you configure that in the docker image, that's not different to bare metal replication.

Same with load balancing in webservers. You can do that! Or you can just have kubernetes handle it. I'm not sure but would expect it to swap images in a way network is optimized - i don't use this kind of software. I just know it's done like this because the bottleneck of modern software is not the local network so its not an issue to have these pieces on different bare metal hosts. And its easier to stay operative if some hosts fail - but you can all solve this by hand.

I work with bottlenecks between network, ssd, ram and vram but if you deploy npm riddled software for >1 Million users well then you may want Kubernetes. Or if you are google and have 10k engineers that need to agree on a standard!

If you can do this yourself with load balancing, replication etc - do it yourself and don't think there's something wrong with that. It's more elegant, efficient - but you have to agree with others how you do it. And that can also be a bottleneck ;)

Still i think you doing it by hand is better so don't worry you are not old you may just have higher standards.

[munchler]

Thank you for this. I probably don't have higher standards, but I do have way fewer than a million users!

[what-is-water]

> Professionally, my experience is that certain software components need to run together on an individual machine (e.g. database server, app server, web server), and then those machines need to be networked in a certain way (e.g. web server talks to app server, which talks to database server).

This (different workload components running on a single machine) is something that kubernetes allows you to disentangle. Kubernetes creates its own network, including cluster internal DNS. Using this you expose e.g. your app servers as a service called my-app, reachable in cluster via my-app.namespace-name.svc.cluster.local.

This targets all containers with a specific label, no matter on which node they run. Server types can be scaled independently, since chances are that the load for each does not scale the same with request volume.

Round robin for DBs does not make much sense, but there are ways to e.g. expose read endpoints with one service, and write endpoints with a different one, with open source tooling which updates the target after a failover.

Kubernetes will automatically keep your services up to date, which means if you increase the replica for e.g. app server the new container will be added as valid target, as soon as it passes ready checks, and if one container fails these checks they are temporarily removed as target. The kubernetes components will also automatically "self-heal" things like a crashed container or a failed node, by restarting the container or rescheduling the workloads on the failed node to a different one, without human intervention.

This is of course very basic, but you can finetune these by e.g. configuring that a specific server type should be spread out, i.e. that only one replica(container) should be scheduled per node, to ensure it stays available if one or more nodes go down. Or add network policies to ensure only the app server is allowed to talk to the DB server.

If your concern is latency between app and db you can have specific config that ensures that your app server containers are only scheduled on nodes where a DB server is already running, and that the traffic from app to DB is always routed to the DB instance that is on the same node (people use similar mechanisms for cloud providers like AWS, where you want to have routing rules ensuring that traffic is always sent to targets in the same AZ, to avoid cross-AZ network charges).

These advanced examples obviously require deeper kubernetes knowledge and are not something one should just try out the first time you deploy kubernetes.

Having worked with more traditional setups I do think it is often easier to configure config like this in the standardized kubernetes API rather than in e.g. nginx config + deployment scripts + idk, systemd-unit. But this point is not "having thousand of nodes" and be half the size of google.

It also depends on your team, if you have an infra team that has a stable way to manage your VMs and apps, all the power to them, replacing them all with k8s experts sure won't give you much. It isn't easy to get an unbiased opinion about when to switch, since you need knowledge of kubernetes and your current infra to make a fair comparison, and kubernetes experts probably want to sell you kubernetes. Using a handcrafted system to distribute a lot of containers over multiple VMs to ensure HA is in general a good sign to evaluate kubernetes ;)

And being on-premise makes kubernetes attractive earlier, since the bigger cloud providers have managed solutions for a lot of things kubernetes helps you with (auto scaling, load balancing, managed container platforms like AWS ECS or Google Cloud Run)