Hacker News new | ask | show | jobs
by Fiahil 1972 days ago
Can you tell us a bit more about the "conflict free merge" ? Is it based on CRDTs ? How does it work ?
1 comments

Yes exactly, it's based on CRDTs and there is a strategy out there to mark the entries with a timestamp to figure out which one are the latest. An object may contain many entries and when they are read by the client they are just compared one by one to find the union, or the latest version.

The beauty of it is that the algorithm decide on how much entries to put in objects to ensure that only the data that is changed is sent on the network and compared on the other client. That's why we call it Condensation.

So, it's like a giant "grow only set" shared between as many nodes as you want and sending diffs on the network? I find it difficult to understand the possible applications* and the link with the article in the readme (https://www.inkandswitch.com/local-first.html).

*: I see that it could be used as backbone for an end-to-end encrypted messaging system, but what would it change for me if I were reading a remote API, running a multi-instance web app or parsing CSVs to train ML models on them ?

It's really not a grow only, there you have immutable data but it expires at some points regarding the rules you want to implement.

The integrity of the data is not shared on the network, each user owns his data, and choose to store it on his desired server. It's really like the email system.

For the article, it really joins the conclusion that you could build google doc, or an IOT system or anything but you will inherent from powerful synchronization, encryption, offline mode and so you can ensure the 7 principles the author of the article characterized.

You could just use a cloud to store massively your data, and pass them through a local server to make sure they are not compromised.

> It's really like the email system

Ah! I see where your inspiration is coming from! It's really interesting when you think of it that way!

I think there is a true need for a decentralized immutable data store (basically get/put/list operations), and it could serve a vast number of use cases. It enables simple algorithmic memoization, complete reproducibility of ML models, and datasets shareability. The only problem is we lack practical solutions. Maybe Condensation could help, or maybe not. In any case, it's good to see more alternative to traditional datastore that are not "git for data".

Yes immutable data stores are already working fine as you can see there https://condensation.io/stores/store/

And yes it really joins your point of horizontal scaling and sharing data in a controlled and secured manner.