Hacker News new | ask | show | jobs
by brown9-2 5578 days ago
Two better approaches come to mind to resolve this:

2. Full-on, properly managed releases like they do in large IT corporations, such as banks, where a "release" is not something you kick off from home via SSH on a Saturday night, but a properly planned effort that involves critical members of the dev team as well as the QA team being present and ready to both test the production system thoroughly and fix any issues that may occur.

What you describe in #2 here sounds like a complete anti-pattern when compared with the idea of continuous deployment and automated verification. This 2nd approach sounds like a huge manual effort.
2 comments
swombat 5578 days ago
It absolutely is, and I'd be surprised to see this kind of effort from any but the most paranoid corporations (like, as I mentioned, banks). Automation and continuous deployment are definitely the way forward.

But even this gargantuan effort is a better option than just "let's deploy and wait for our users to tell us if anything has gone wrong".

link
brown9-2 5578 days ago
But even this gargantuan effort is a better option than just "let's deploy and wait for our users to tell us if anything has gone wrong".

To be fair it sounds like in the original article that they did do some verification that things were working after the deployment. However for some reason their verification tests didn't reveal the presence of a real bug.

Even in a more gargantuan system, it's possible to have tests that give false positive results.

Everyone will screw up releases at some point, the key is to be able to learn from them and get better.

link
peterwwillis 5578 days ago
If you're making a big change you first cut a CR and get approval of any teams involved. At change time, everyone knows they need to be on-call if something breaks, preferably in a live chatroom.

The rest of the time devs should just deploy when they think the code is ready and have tested it on a like-production box. They then manually verify the change worked. You use automated monitoring to ensure when something does break you are notified immediately.

link