[eysi]

My team has been developing against a fully remote environment (K8s cluster) for some years now and it makes for a really powerful DevEx.

Code sits on our laptops but live syncs to the remote services without requiring a Docker build or K8s deploy. It really does feel like local.

In particular it lets us do away with the commit-push-pray cycle because we can run integ tests and beyond as we code as opposed to waiting for CI.

We use Garden, (https://docs.garden.io) for this. (And yes I am afilliated :)).

But whether you use Garden or not, leveraging the power of the cloud for “inner loop” dev can be pretty amazing with right tooling.

I wrote a bit more about our experience here: https://thenewstack.io/one-year-of-remote-kubernetes-develop...

[jayd16]

Kind of interesting to think that CI is significantly slower in practice and both systems need to be maintained. Is it just the overhead of pushing through git or are there other reasons as well?

[eysi]

The way we do things is that we build everything in the cloud and store in a central container registry. So if I trigger a build during dev, the CI runner can re-use that, e.g. if it’s needed before running a test or creating a preview env.

Similarly if another dev (or a CI runner) triggers a build of one of our services, I won’t have to next time I start my dev environment. And because it’s built in the cloud there’s no “works on my machine”.

Same applies to tests actually. They run in the cloud in an independent and trusted environment and the results are cached and stored centrally.

Garden knows all the files and config that belong to a given test suite. So the very first CI run may run tests for service A, service B, and service C. I then write code that only changes service B, open a PR and only the relevant tests get run in CI.

And because it’s all in prod-like environments, I can run integ and e2e tests from my laptop as I code, instead of only having that set up for CI.

[mewpmewp2]

You would need a very perfect and flexible CI system in place that wouldn't need to rebuild anything it doesn't need and only run the tests you want or only recently failed tests etc.

Many CI systems would spin up a new box instead of using persistent so likely have to rebuild if no cache, etc.

So basically I would say most of the overhead is in not having a persistent box with knowledge of last build or ability to choose what to run in there, which pretty much just equals to local capabilities.

[peanball]

Often you also have the CI system designed in a way to verify a “from scratch” build that avoids any issues with “works on my machine” scenarios due to things still being cached that shouldn’t be there anymore.

[jayd16]

Having persistent boxes with sticky sessions seems seems pretty achievable.

[8n4vidtmkvmk]

I tried Garden briefly but didn't like it for some reason. DevSpace was simpler to set up and works quite reliably. The sync feature where they automatically inject something into the pod works really well.

[eysi]

DevSpace is a great tool but it’s bummer you didn’t like Garden.

Admittedly, documentation and stability weren’t quite what we’d like and we’ve done a massive overhaul of the foundational pieces in the past 12 months.

If you want to share feedback I’m all ears, my email is in my profile.