Hacker News new | ask | show | jobs
by ChrisArchitect 1490 days ago
957 hours. Pretty crazy. Can't think of another 'outage' with that kind of length on it in awhile or ever.
4 comments

Yeah, just last week I was thinking 'this is pretty amateur hour' as I spent almost a whole morning bringing us back up..

(We are.. a tiny fraction of Heroku, in terms of anything you like - it's excusable IMO that it was an untested procedure not smooth etc., small team with MVPs to ship.)

In 957h I would think you can start to think about bringing on a specialist on contract (or implement the new GH App based still-future version mentioned) if the permanent team can't figure it out / don't have capacity! It's not good for reputation, surely, I have to imagine it was considered low priority rather than something they actively tried but failed to fix for so long, but I don't think that's a good look, even if metrics show it's little-used or only by free tier or whatever.

The remaining engineers had to deal with 900 hours of replying to recruiter spam. Please excuse them. /s
Atlassian had a similar one.
Only the github connect was broken, automatic deploy. Just push to heroku also, when you push to github.

I just re-enabled the Github connect, re-enabled Automatic Deploy, and removed the `git push heroku` line. 1min.

Sure, but that means deploying was broken for a large number of people, and that they had to spend a non-trivial amount of time setting up an alternative deployment method. For us it wasn't too bad as we already had external CI pipelines (but it still took me most of a day to sort it out). But if you were using heroku for CI then it would have been a much bigger issue.