[karka91]

Assuming this is about web dev.

Nowadays - flip a toggle in the admin. Deployments and releases are separated.

Made a major blunder? In kubernetes world we do "helm rollback". Takes seconds. This allows for a super fast pipeline and a team of 6 devs pushes out like 50 deployments a day.

Pre-kubernetes it would be AWS pipeline that would startup servers with old commits. We'd catch most of the stuff in blue/green phase, though. Same team, maybe 10 deployments a day but I think this was still pretty good for a monolith.

Pre-aws we used deployment tools like capistrano. Most of the tools in this category have multiple releases on the servers and a symlink to the live one. If you make mistake - run a command to delete the symlink, ln -s old release, restart web server. Even though this is the fastest rollback of the bunch the ecosystem was still young and we'd do 0-2 releases a day.

[Shalle135]

Why not canary releases? You can load balance for example 1% of the traffic to the new deployment and see if you experience any issues. If you do - you just change the loadbalancer to use the known good pods.

[karka91]

Honestly, we haven't considered that. Breakage rarely reaches production, rollbacks are quick, a process orchestrator retries most of the stuff and because of that we do not have a problem worth solving with "canary deployments". Note that we do sometimes carry out "canary releases" via toggles.

[pojzon]

How you take care of DB updates when using cannary deployments ? For example those which are not backwards compatible ?

Ps. Releases are about building new versions of code packages. Deployments about pushing them out to environments.

[karka91]

This depends a lot on the databases used and flexibility of the code that's accessing the data. One method is to deploy in multiple stages and to use views. In pre-deploy you create a view with schema/data needed for the release. In deploy stage you roll-out the canary code. In post-deploy stage remove either the old data/schema on success or new data/schema on failure. It's quite an overhead to implement and maintain this process.

[bpicolo]

Best way is to not write forwards / backwards incompatible database changes.

[korijn]

> Nowadays - flip a toggle in the admin. Deployments and releases are separated.

Would you mind explaining this a little further? How does the separation allow you to flip a switch in the admin?

[rgoulter]

AFAIU this is making use of feature flags.

Uploading new code to a server where the new-code is behind a disabled feature flag means the program hasn't changed. The feature flag can be enabled when it's suitable. (This could even be 'rolled out' to only a subset of users; e.g. testers, internal, 10% of users, etc.)

deploy = put the code on the server, release = enable the feature flags