Hacker News new | ask | show | jobs
by throwusawayus 1552 days ago
from that article it sounds like they are mostly doing "functional partitioning" (moving tables off to other db primary/replica clusters) rather than true sharding (splitting up tables by ranges of data)

functional partitioning is a band-aid. you do it when your main cluster is exploding but you need to buy time. it ultimately is a very bad thing, because generally your whole site is depenedent on every single functional partition being up. it moves you from 1 single point of failure to N single points of failure!

2 comments

>> functional partitioning is a band-aid.

I disagree, functional partitioning is not a band-aid, but an architectural changes that in the end can reap much more benefit than simple data sharding.

>> your whole site is dependent on every single functional partition being up. it moves you from 1 single point of failure to N single points of failure!

Not necessarily, it can also be that only some parts of your site are dead while others work perfectly fine.

too idealistic. invariably some team (or usually many teams) don't properly gate some critical-path logic, they depend on some functional partition always being online and then boom much larger blast radius than intended

then they fix it in post-mortem but pattern just repeats. i have seen it so many times! used to be much worse in the earlier days of the cloud when VMs would go poof more often

Towards the end:

> In addition to vertical partitioning to move database tables, we also use horizontal partitioning (aka sharding). This allows us to split database tables across multiple clusters, enabling more sustainable growth.

yes i know, the fact that this is a small blurb at the bottom of the article (which is largely about functional partitioning) exactly proves my point