[derefr]

File system snapshots are a system that have the analogous desired properties for files; but filesystem snapshots are actually quite heavyweight, because they deal with dirents, inodes, extents, etc. CoW filesystem snapshots are designed for ops-task-granularity usage, e.g. daily backups; not for per-transaction historical archiving. CoW filesystems tend to fall over once you get to 100K snapshots. (I tested!) A database that took a snapshot after every CQRS/ES transaction, could be expected to potentially have billions of snapshots.

A system that did its snapshots “inline” to itself, by e.g. managing a pool of pages with a free-list the way LMDB does — but where Txs ultimately add a new version to the root bucket as a snapshot, rather than replacing the root bucket page with themselves — would get a lot closer to allowing one to have at least tens-of-millions of snapshots online. At that point, to achieve a billion snapshots online, you “only” need to shard your timeline across a cluster of 100 nodes.

This is precisely one of the experiments I’m trying. :)