Hacker News new | ask | show | jobs
by jitl 812 days ago
(I work at Notion)

Our shard key - Workspace ID - is a UUIDv4 so there’s a pretty high number of orgs per shard without conflict.

1 comments

Hey Jake, I meant capacity per shard in this case, not exhausting the IDs. Any potential solutions for that or is that not an immediate challenge?
Gotcha. With our shard strategy we add more capacity either by scaling up nodes (very easy), or by resharding - adding nodes to the cluster and putting fewer shards on each node.

We recently did a reshard from 32 nodes / 15 shards per node -> 96 nodes / 5 shards per node. That puts us in footing to scale up for a while before we need to reshard again. This is a pretty smooth process, and each time we scale out we get much more scale up capacity.

Our shard logic is very simple static assignment based only on the Workspace ID. If we wanted to add workspace created time routing, we'd need to starting plumbing that information around the system in ways that are slightly annoying. Probably the move would be to re-key the Workspace table to use a date-embedding UUID format.

https://www.notion.so/blog/the-great-re-shard