| If you're working at a large company and downtime is extremely expensive, this checklist is a good guide. Otherwise, if you have good test coverage, you can get by with something simpler. It's super rare to have a breaking change in go. We do quarterly upgrades of all services in a monorepo (about 20-30). The steps are basically this: - Upgrade all dependencies to their latest versions, fixing build and test breaks (I read release notes for Go, but not for dependencies) - Look for deprecated packages and replace them - Upgrade all toolchains, including CI/CD containers, go.mod, etc. - Run all tests - Deploy to the test environment and make sure everything is green - Deploy to staging and do some sanity checks - Deploy to prod, keeping an eye on metrics for an hour or two We're on k8s and the state of all clusters (i.e. which images are running) is tracked in git, so a rollback is just git revert + apply. In practice, after about four years of this, we've seen maybe a dozen build breaks, and I can only remember one regression caused by a breaking change in a library[1]. [1] https://github.com/golang/go/issues/24211 |
Since it's over the total number of builds for 20-30 services, it should not be that bad; instead, sometimes there happened a completely uneventful upgrade of everything!