Hacker News new | ask | show | jobs
Backblaze S3 API is having some downtime (s3.us-west-002.backblazeb2.com)
32 points by doctorfoo 2044 days ago
9 comments
atYevP 2044 days ago
Yev here - we're investing the S3-endpoint issue - apologies for delay in response here, it took them a bit of time to wake me up.

*Edit -> we've identified the fix and are rolling it out, you should see S3-Endpoint service resuming shortly!

link
mahesh_rm 2044 days ago
Happy to know who to bother the next time it happens! :P EDIT: But really, get a simple b2s3api status page up, doesn't need to be anything fancy..
link
atYevP 2044 days ago
Well, hopefully there won't be a next time - but also, bothering me doesn't help debug or resolve the problem, our Ops/Tech team do that :D
link
OJFord 2044 days ago
I was baffled that I couldn't find a status page at all, but I assumed I was just failing to find it.

> Well, hopefully there won't be a next time

Are you seriously saying that not only is there not one, but you're against having one because you hope the status is always green?

link
atYevP 2044 days ago
No, that comment was edited after I had responded - we're not against status pages - they are on the roadmap, but unfortunately not implemented yet.
link
autorun 2032 days ago
you just need to update your pixelated loading spinner, everything else is great, thanks for so much, sorry for so little
link
OJFord 2044 days ago
Ah, glad to hear, thanks!
link
doctorfoo 2043 days ago
Thank you, and I do love your service. It would be nice to see a post mortem on what went down. /goes to sleep
link
nhoughto 2044 days ago
Looks like Europe is down too?

https://s3.eu-central-003.backblazeb2.com/

link
koreanguy 2044 days ago
https://bgp.he.net/dns/backblazeb2.com#_website

This site can’t be reachedCheck if there is a typo in backblazeb.com. If spelling is correct, try running Windows Network Diagnostics. DNS_PROBE_FINISHED_NXDOMAIN

link
nhoughto 2044 days ago
But non s3, b2 api is up

https://f002.backblazeb2.com/file/blah

https://f003.backblazeb2.com/file/blah

link
akchou 2043 days ago
Today's task: starting working on moving back to aws. Storage savings cant compete with the cost of developer time and downtime even when we store a petabyte.
link
OJFord 2044 days ago
Yes! Glad it's not just me. And no status page. (?!)
link
jsiepkes 2044 days ago
I also can't find a status page and not so much as a peep from Backblaze anywhere. I pinged them on Twitter [1] to no avail. I like Backblaze but this has been going on for 3 hours now. You can't be down for 3 hours without any communication about it...

[1] https://twitter.com/JasperSiepkes/status/1329347119774375937

link
OJFord 2044 days ago
From the support page, I get the impression nothing will happen until 9am Pacific. Tickets have a 1d response time, and chat is Pacific working day with a lunch break.

Bottom of https://www.backblaze.com/help.html

I think they're all sound asleep, blissfully unaware of the day they'll wake up to!

link
slice-beans 2044 days ago
It doesn't reflect well on their monitoring/on-call setup that everyone else was paged before them, assuming they are indeed unaware.

Perhaps the first they'll know of the problem is when they read it on HN

link
doctorfoo 2044 days ago
yup, here desperately migrating to the regular b2 API...
link
zhan_eg 2044 days ago
Well, it looks like it is in the time frame of their scheduled maintenance https://www.backblaze.com/scheduled-maintenance.html -- Disregard, I got AM and PM confused :(
link
OJFord 2044 days ago
No, I saw that, but that says 3 pm not am.

Pacific time is several hours behind me in the UK and it's 11am here.

I don't think it was even Thursday yet Pacific time when this started.

link
jsiepkes 2044 days ago
Can't be because this has been going on for over 3 hours now. The timeframe is only 1 hour.
link
mahesh_rm 2044 days ago
Argh. I believed I was going insane. How can this be the only B2 status page?
link
chima3ra 2044 days ago
well, what can u expect , 4 times cheaper than AWS, 4 times less stability and professionalism.

I am now wandering if my data is safe there. I think it will be smart to keep a copy in aws glacier.

link
doctorfoo 2044 days ago
Feeling kind of the same... I just completed moving my entire app and all user data from AWS. And here we are!
link
manojlds 2044 days ago
How prominent is usage of Backblaze and the S3 API in specific? Is HN going to be status page for any random service?
link