Hacker News new | ask | show | jobs
by sgrenfro 4741 days ago
When the delete button affects a single row in a single database table, it's indeed really simple. Even if it's a few rows on a single database, you can use transactions. Either way, the deletion succeeds or you surface an error message.

But for a large social site, it starts to become a more interesting distributed systems problem.

Even for a simple story with 4 likes, 12 comments, and likes on several of the comments, the data is sharded across several databases. Simple transactions no longer work. One option is looking for scalable distributed transactions. A more likely option is making sure deletions can be interrupted and restarted without losing state.

Now imagine a public figure whose post has tens of thousands of comments and hundreds of thousands likes. You may not be able to load all of those rows in a single request let alone delete them synchronously. Instead, you have to be able to process the deletion incrementally, reading and deleting some of the data then later picking up where you left off the last time you were interrupted.

Interestingly, the order of these operations can also be important. If you're interrupted between deleting a comment and deleting the likes on that comment, will you still be able to find the likes when you restart?

We've put a lot of work into a deletion framework that deals with these and many other issues. We're hoping to share more details about it in the future.

1 comments

So you're telling me that you/they manage to commit inserts and updates to your/their sharded databases reliably but deletes conveniently fail.
Well comment tree deletion is a harder operation than post creation or comment creation or liking.

Each of the steps in building up the tree of comments, likes, etc. is a a couple writes at most. If that fails, you're left in a safe state. You tell the user it failed and carry on.

Deleting a post and the comments on it is harder. First, you have a potentially unbounded amount of DB rows to delete across an unknown number of shards. Second, you now have to worry about the order in which you delete things, else you risk being unable to retry in case one of these deletes fails. Third, unlike the inserts and updates, you can't just give up partway, as that's not a safe state to end in.