[giobox]

Kubernetes and docker solve different problems. One is a container runtime, the other is a container orchestration tool. A mega-corp using Kubernetes can still use Docker, the two can work together.

"Docker compose" (not the same thing as docker) works great at single machine scale/local development environments, but isn't really designed to scale much beyond this to production environments, multiple servers and data centers etc, which Kubernetes is. This isn't to say you couldn't deploy something to production with compose, its just not very likely outside of small personal projects - there are heaps of features in Kubernetes that simply don't exist in compose.

Generally you'd typically find a docker compose configuration for easy local development environment deployments, a Kubernetes configuration for managing the production environments, although there are no hard and fast rules here. Compose generally works best where the services fit all on the same box, which is rare for a business of almost any size in production, but common for local dev work.

I also prefer Compose for personal projects and local development, but it simply wouldn't work at any place I've worked for production deployments.

[selcuka]

Note that one can also use a lightweight Kubernetes distribution such as Minikube [1] during development so that the workflow is similar for both development and production.

[1] https://minikube.sigs.k8s.io/docs/start/

[mmcnl]

But this often not necessary, because if you can run a production Kubernetes cluster, you can also run a dev Kubernetes cluster, so no need to run a local Kubernetes distribution.

My workflow typically looks like this:

1. Run app locally during dev without Docker

2. Build and run Docker image locally with Docker compose

3. Deploy to development Kubernetes cluster

4. Deploy to production Kubernetes cluster

[selcuka]

> if you can run a production Kubernetes cluster, you can also run a dev Kubernetes cluster

When you have many developers, the cost for maintaining one dev cluster per developer quickly goes up. One cluster for all developers can be used for testing/staging, but not for development.

Minikube is a replacement for your step (2), except you can now use your existing tools (kubectl etc) instead of docker-compose.

[KronisLV]

> "Docker compose" (not the same thing as docker) works great at single machine scale/local development environments, but isn't really designed to scale much beyond this to production environments, multiple servers and data centers etc, which Kubernetes is. This isn't to say you couldn't deploy something to production with compose, its just not very likely outside of small personal projects - there are heaps of features in Kubernetes that simply don't exist in compose.

In short:

  - if you need to deploy containers on a single node ad-hoc, then you want Docker or a similar runtime (e.g. Podman)
  - if you need to deploy containers on a single node with a deployment descriptor, then Docker Compose is a good option, due to its simplicity
  - if you need to deploy containers across multiple nodes with a deployment descriptor, then you want some sort of an orchestrator

I'd say that going from Docker Compose to Docker Swarm is the first logical step, because it's included in a Docker install and also uses the same Compose format (with more parameters, such as deployment constraints, like which node hostname or tag you want a certain container to be scheduled on): https://docs.docker.com/compose/compose-file/compose-file-v3... That said, you won't see lots of Docker Swarm professionally anymore - it's just the way the job market is, despite it being completely sufficient for many smaller projects out there, I'm running it in prod successfully so far and it's great.

Another reasonably lightweight alternative would be Hashicorp Nomad, because it's free, simple to deploy and their HCL format isn't too bad either, as long as you keep things simple, in addition to them supporting more than just container workloads: https://www.hashicorp.com/products/nomad That said, if you don't buy into HashiStack too much, then there won't be too much benefit from learning HCL and translating the contents of various example docker-compose.yml files that you see in a variety of repos out there, although their other tools are nice - for example, Consul (a service mesh). This is a nice but also a bit niche option.

Lastly, there is Kubernetes. It's complicated, even more so when you get into solutions like Istio, typically eats up lots of resources, can be difficult to manage and debug, but does pretty much anything that you might need, as long as you have either enough people to administer it, or a wallet that's thick enough for you to pay one of the cloud vendors to do it for you. Personally, I'd look into the lightweight clusters at first, like k0s, MicroK8s, or perhaps the K3s project in particular: https://k3s.io/

I'd also suggest that if you get this far, don't be afraid to look into options for dashboards and web based UIs to make exploring things easier:

  - for Docker Swarm and Kubernetes there is Portainer: https://www.portainer.io/
  - for Kubernetes in particular there is Rancher: https://www.rancher.com/ (they are also the people behind K3s)
  - Kubernetes also has its own dashboard, but it's a bit less nice in my opinion: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
  - Hashicorp Nomad comes with its own web UI as well, though I recall it being a bit barebones: https://developer.hashicorp.com/nomad/tutorials/web-ui

In the terminal, there are also a few useful projects:

  - for Docker, there is ctop: https://github.com/bcicen/ctop
  - for Kubernetes, there is k9s: https://k9scli.io/

As for a desktop UI, there are a few options too:

  - Docker Desktop: https://www.docker.com/products/docker-desktop/
  - Rancher Desktop: https://rancherdesktop.io/
  - Podman Desktop: https://podman-desktop.io/

You don't strictly need any fancy tools like that and Docker/Podman CLI, or kubectl can be enough, but sometimes a more visual look and being able to click around can be nice.

[totalhack]

I use swarm for smaller projects and am happy with it, though I know it's out of fashion now. I may try Nomad next time around. If I needed something more advanced or with more scale I am probably in a situation where a managed service would be within the budget.

[_huayra_]

Any of the options lighter-weight that Kubernetes is usually good to start with, but for anyone already running these solutions (e.g. docker compose), learning Kubernetes can be a valuable job skill (because like you said, it's for mega-corp type environments).