Hacker News new | ask | show | jobs
by zzz61831 2138 days ago
> Say 100 nodes edit a doc. To drop a tombstone, without a central server we need 99 acks. Even with a server, the server keeps an ever-growing list of ops unless it uses some garbage collection techniques. And as stated by OP those techniques have their own problems.

Here's how it could be done (very roughly):

When a client makes a modification, it creates an op representing that modification and includes a list of server identifiers to send the op to. Then queues the op. Once op is sent and ACKed by at least one of the servers the op is dropped from the queue.

Servers receiving the op synchronize the op with each other, ACKing each other and removing ACKed servers from the list. They also synchronize with the clients, sending the op to each client and asking clients to ACK to all servers. Obviously there are multiple ways to do it and it can be almost as efficient as theoretically possibly, but of course you can't avoid some metadata for each client associated with a particular document even though it doesn't have to be kept in server memory, it could be stored on disk.

> I'm talking about structures with semantic meaning.

Semantic meaning generally doesn't map to any data structure directly, CRDTs are not an exception here either.

1 comments

> Then queues the op. Once op is sent and ACKed by at least one of the servers the op is dropped from the queue.

Unfortunately, it's a lot more complicated than this.

First, you can't wait until just one peer has ACKed -- you need to wait until all of them have. Any peer that hasn't ACKed could still send an op based on the one you want to delete, and then you wouldn't have enough information to merge the new op.

Second, you can't just wait until all peers have ACKed an op, you need to wait until no new ops depend on that op. An op could depend on an op even though it has been ACKed by everyone.

Third, a peer could go offline for a while, and then come back online. But if you are waiting for all peers to ACK an op, you might never get a situation where all peers are online at the same time.