[sdairs]

Hey, Al from Tinybird here (co-author of the post). We've made up to 20 production releases per day some days. It's transparent to our users, they aren't even aware the upgrade is happening, there's no upgrade button to hit, there's no downtime. We release often because we release small and fast. It's not like those 20 releases are always fundamentally changing the product. We would rather fix a minor bug or two and get that out to our customers ASAP, than hold on to it for a few months and drop a huge change. In a vast majority of cases, a user won't even consciously notice something changed.

Doing this kind of fast iteration has its risks, but it has its benefits too. We de-risk it, in part, by having extensive CI, which is why it was so important to us that the CI is fast & reliable.

Delivering larger, less-frequent updates has its own risks. You're not practising your release process as frequently, so it's a much bigger event. You're pushing many, many more changes in one go, so there's a lot more surface area for something to go wrong, and rolling it back is a much bigger job. And dropping many/bigger changes to the user experience is much more noticeable.

Again, this isn't the right process for everyone, but it works for us and its how we've managed to build a product that delivers value to our users.

[chrisandchris]

It's interesting how you differ between "deploy ASAP, within half a day" and "keep the fix a few months back".

Like, is there nothing in between? Like once a week, once every other week?

If you would have to estimate, wouldn't be there less bugs if you would deploy less fast (and use this time for validation)?

[sdairs]

Sure :D

I've worked on products across the spectrum, Enterprise software that releases twice a year, smaller stuff that ships once a month, and now a SaaS that releases many times a day. I don't have the data to compare, but I genuinely don't believe the rate of bugs was materially different between any of them.

We make faster changes, but the changes are much smaller, and so the surface area to test is smaller. The more changes you make, the more time you need to validate. If you make a weeks worth of changes, you're going to need an appropriately longer validation cycle than if you make 1 change. It scales, and in my experience, many products with slower release cycles aren't appropriately scaling up the validation time to match.

That said, this doesn't necessarily mean we're writing code and 30 minutes later its in production...there's still an iterative dev cycle with lots of validation happening...but if something is ready to go, ship it!

[Scubabear68]

Thanks for the response Al!

Very interesting. I agree different strokes for different folks, you guys seem to be on the extreme end of CI/CD.

Have you done any sort of analysis you could share on what it costs to release up to 20 times per day?

[sdairs]

We do track the real $ cost of time & materials, but tbh I don't think it's anything too exciting. I'll see what we can share!