Hacker News new | ask | show | jobs
by sseveran 4133 days ago
I am not quite sure where to begin. I read your wiki. The casual disregard for years of distributed computing research struck me as a bit scary. It would be fine if the website did not claim "Data integrity is now a breeze." but it does. It turns out data integrity in a distributed setting is actually quite a hard problem given the number of boundary cases that exist. For that reason experienced practitioners building distributed algorithms start with proofs, not try to come up with them later.

From reading your page on conflict resolution, which is quite light on details, it seems like you want to have a transaction log but unless the data is purely commutative that is impossible in an AP environment.

If you are going to reference a "Hypothetical Amnesia Machine" it would be helpful to at least define it. Looking at the code you hand someone two versions of their data and ask us to merge it.

It's nice that you are enthusiastic but it might be nice to build some proofs (or use someone elses) before making claims that our distributed state problems are solved.

1 comments

This person (in the comments below, please upvote him), and my reply, best addresses the most important questions: https://news.ycombinator.com/item?id=9077969

But to address your other comments:

Why can't a transaction log be commutative? You just have to make sure to be explicit about the ordering of events (like by using linked lists or progressively incremented hashes). This is the realm for CRDTs and stuff though, which GUN core doesn't touch.

Yes the HAM deals with merging any two logs/streams/history, any two snapshot/states, but also merging any log/stream/history with any snapshot/state.

This is important, because it allows you to merge more than just two, you just have to do it serially (in any order). That merge algorithm guarantees the deterministic resolution (see the comment link I posted above).

It relies on the data being commutative. If the data is not commutative then we need, wait for it, drum roll please, some consistency, which by your own admission is not provided. The consistency is the hard part which is why there is an entire field of study on just this problem.

I fail to see how your merge algorithm is deterministic in case of failures.

You -cannot- have a distribute database without consistency. Prove us ALL wrong, please.
Maybe I wasn't clear... GUN does have Eventual consistency, but not strong/global consistency.

Aka GUN is AP and Eventually consistent. You manually at the application layer can decide to lock, sacrificing Availability, and get strong consistency.

The merge algorithm works in an Eventually consistent case, but obviously is too naive for global Consistency, you'd need some form of consensus.

Or does that not address your comment?