[fishfacemcgee]

Good postmortem from BB. Crappy situation for them to have to be in since it was all Adobe's fault, but I think they handled it well. They didn't off-load problem-solving to Adobe, but made it clear what the root cause of the bzvol error was.

Meanwhile, I'd love to see the postmortem from Adobe. What file(s) were they expecting to delete and why did they feel it prudent to not do a strict/stricter check for the files/directories to delete? Presumably some sort of "adobe" directory, but I would hope to at least find out that they couldn't reliably expect the folder to be a specific name and the issue was that their searching was just too loose.

[mirimir]

They obviously made no check. The goal was probably to delete all Adobe configuration files. And somehow the first hidden folder in home became a target. Because .adobe is obviously going to be first, right?

[ajanuary]

Deleting the first directory seems like a harder problem than deleting a folder called .adobe.

[mikeash]

I don't know about that. It's harder to do it right, but they probably didn't bother with doing it right.

For example:

    ls / | head -n 1 | xargs rm -rf

[ajanuary]

That's easier than

    rm -rf /.adobe

?

My point being that things indicate a more interesting problem they were trying to solve than just deleting a directory. What that actually was, and how that lead to such an awful solution, would be interesting to know.

[mikeash]

I agree it's pretty far fetched. I'd love to know what the real story is too.

[kevb]

I wonder if it was something like .adobe-A3DF0910 that they were targeting, so they couldn't completely hard code the path and neglected to hardcode the prefix. Or maybe it was a purely randomly named directory without any prefix.

[mirimir]

Maybe they were targeting .adobe* in home, and assumed that it'd be first, but didn't account for that possibility that it'd be missing. That would have been insane, I admit. But we do know that initial dot directories were getting hit.

[atYevP]

Yev from Backblaze here -> To their credit Adobe did put out this blog post/update -> http://blogs.adobe.com/adobecare/2016/02/12/creative-cloud-d....

[durandal1]

Why is BB writing to ~/ instead of ~/Library/Application Support/BackBlaze?

[brianwski]

Brian from Backblaze here, here is an explanation I typed in elsewhere:

I'm the guy who decided to put things in a folder at the root level of drive. Originally this was a system for EXTERNAL drives that get plugged in then unplugged, then come back later. We need to know what state the backup is in related to that drive, and if the customer has several external drives that come and go we need a unique ID for each one. So I created a top level folder on the external drive called ".bzvol" (hidden) and then placed two files inside of it, a README that explained what the folder was and who created it and why you should not delete it, and a 100 byte XML file that had the drive's unique id AND ALSO the identifier of the backup that owns this drive (some customers have two computers both backed up by Backblaze and they carry a hard drive between them).

There are many designs that would work, in retrospect a better design might be to have used the drive's internal serial number (which is globally unique) and maybe a little mapping database either stored on the Backblaze website or somewhere down under /usr/local or /Library that maps the globally unique drive back to the backup it is associated with.

That's the problem with developing software. After you have worked on a problem for 8 solid years, you are finally qualified to BEGIN working on it and should rewrite it from scratch. :-)

[durandal1]

Thanks for the reply - I completely understand that historical design design sometimes leads us to end up in non-ideal situations.

[elcct]

Maybe they hired a bunch of Stackoverflow driven developers and it was a result of a copy paste