Hacker News new | ask | show | jobs
by itake 476 days ago
> - Determining which database each piece of data lives on (as opposed to using partitioning keys which do that automatically)

Most sharding databases use consistent hashing.

> - Manually rebalancing data, which is often difficult and error-prone

not really. When you setup the database, you choose a highly divisible number of shards and then consistent hashing to spread the data across the shards. Each server hosts N-shards where N changes as your data (and server count) grows

> - Adding partitions manually as the system grows

Not really. Just choose a reasonably high number of shards and divide them across your servers.

> - (Anecdotally) Higher operational costs, since matching node count to workload is tricky

This could be true, but also, there is overhead to managing a ton of SQLite databases too. I think there are tradeoffs here.