[Ozzie_osman]

Curious if you had any tables that were "universal" and weren't shardable, global stuff that isn't workspace-specific. Would these be replicated across all shards? Or just stored in some special db (in which case joining to them might be difficult).

[garrettf]

Plenty! But not significant compared to page content in terms of storage or load. That unsharded data is stored in separate unsharded databases today, but each table could be sharded later using a different shard key (often user id).

Luckily we did not have many join queries prior to sharding. The few cross-database joins were trivial to implement as separate queries in application logic.

Our main concern here was referential consistency: if you need to write to multiple databases at once, what happens if one of your writes fails? This was not a problem in practice for us since (1) our unsharded data is relatively static and (2) there are very few bidirectional pointers between data across databases.

Long term there are more interesting problems to solve when distributing unsharded data globally. However, given that our unsharded data is less dynamic and consistency is less critical, we have many levers we can pull here.