[ojame]

We host our app on Heroku - the core service and a few microservices. The heroku dashboard is down, and we can't use the cli due to the in-browser auth callback that it needs (which is also down).

The status page itself is either saying nothing is wrong, or points to an error page[0]. The incident itself[1] hasn't been updated, which is pretty frustrating.

We can't submit a support ticket because, well, it requires the authentication procedure as well.

We use worker queues, and the queues are getting blown out because heroku can't action anything. We're having our microservices yo-yo now, which suggests things are getting worse, now better.

I've always been a huge Heroku advocate, but the last 5 years have been death by a thousand cuts.

0: https://status.heroku.com/error 1: https://status.heroku.com/incidents/2822

[ageitgey]

This feels like the last paper cut for me. It's easy enough to keep using Heroku because it's "easy for the team," even though it is overpriced. But when they can't even acknowledge an incident and there is no way to log in and file a ticket, it feels like they have failed.

[jrochkind1]

That status incident page at https://status.heroku.com/incidents/2822 is currently taking nearly 60 seconds to load for me, with a spinner.

Which is odd, heroku I'd think would be pretty good at keeping it's status page infrastructure separate enough to stay up. Must mean something pretty fundamental in their architecture is malfunctioning. :(

but when I am able to see the error page, it did say "Heroku continues to investigate and remediate an issue with intermittent outages" -- I would say it is acknolwedged. Yes, that message is 3 hours old. The fact that it's taking them over 3 hours to fix is disturbing, but getting contant progress communication isn't really urgent for me -- I know they know about it, I know they are working to fix it, I'd like them to fix it _quicker_ but I don't need a play-by-play, "can't even acknowledge an incident" is NOT a problem being exhibited, it's acknowledged.

We'll wait and see what it was. A good retrospective write-up goes a long way to increasing many people's confidence, including mine.

[senfiaj]

I wonder whether heroku is some kind of monolith, if something crashes, the whole thing crashes.

[jrochkind1]

In general, I was getting death-by-a-thousand-cuts pessmisim about heroku, but I'm really optimistic about the new Fir stack, which will fix some of my biggest complaints.

Stability definitely still matters though, of course.

[senfiaj]

The cli doesn't seem to work anyways, I can't access or restart dynos.