[cyberax]

I tried to use K8s several times, and I just can't make it work. It's fine as a deployment platform, but I just can't justify its complexity for local development.

We're using Docker/Podman with docker-compose for local development, and I can spin up our entire stack in seconds locally. I can attach a debugger to any component, or pull it out of the Docker and just run it inside my IDE. I even have an optional local Uptrace installation for OTEL observability testing.

My problem is that our deployment infrastructure is different. So I need to maintain two sets of descriptions of our services. I'd love a solution that would unify them, but so far nothing...

[stackskipton]

I wouldn't use K8s for local development unless you have some system where there is a dev cluster and you can route traffic for particular pod to your local workstation.

Docker Compose for local development is fine. If your K8s setup is crazy complex that you need to test it locally, please stop.

[fragmede]

Tilt? Skaffold? configuration isn't free, but a debugger on a local k8 cluster that's at least somewhat representative of prod is pretty handy once you do.

[cyberax]

I tried Tilt, but it's still too complicated. For example, we have a computer vision model that is a simple Python service. When developing on macOS, it's not possible to use GPUs inside containers, so we need to run it locally on the host.

It's trivial with my current setup, but not really possible with Tilt.

[ghthor]

I know this pain, though we’re running nomad not k8s as our cluster control plane. But local devs are stuck with docker-compose, so 2 different configurations for running locally versus in the production environment.