[Sammi]

All this recent hype about sync engines and local first applications completely disregards conflict resolution. It's the reason why syncing isn't mainstream already and it isn't solved and arguably cannot be.

Imagine if git just on its own picked what to keep and what to throw away when there's a conflict. You fundamentally need the user to make the choice.

[aboodman]

Zero (zerosync.dev) uses transactional conflict resolution, which is what our prior product Replicache and Reflect both used. It is very similar to what multiplayer games have done for decades.

It is described here:

https://rocicorp.dev/blog/ready-player-two

It works really well and we and our customers have found it to be quite general.

It allows you to run an arbitrary transaction on the sever side to decide what to do in case of conflicts. It is the software equivalent of git asking the user what to do. Zero asks your code what to do.

But it asks it in the form of the question "please run the function named x with these inputs on the current backend db state". Which is a much more ergonomic way to ask it than "please do a 3-way merge between these three states".

Conflict resolution is not the reason why there has not been a general-purpose sync engine. None of our customers have ~ever complained about conflict resolution.

The reason there has not been a general-purpose sync engine is actually on the read side:

  - Previous sync engines really want you to sync all data. This is impractical for most apps.

  - Previous sync engines do not have practical approaches to permissions.

These problems are being solved in next generation of sync engines.

For more on this, I talk about it some here:

https://www.youtube.com/watch?v=rqOUgqsWvbw

[probabletrain]

I think with good presence (being able to see what other users are doing) and an app that isn't used offline, conflicts are essentially not a problem. As long as whatever is resolving the conflicts resolves them in a way that doesn't break the app, e.g. making sure there aren't cycles in some multiplayer app with a tree datastructure. Sounds like Zero has the right idea here, I'll build something on it imminently to try it out.

[Sammi]

Agree that if you don't have offline support, then conflict resolution is such a minor issue that you can just do "last write wins" and call it a day.

[Sammi]

"It is the software equivalent of git asking the user what to do. Zero asks your code what to do."

You are asking the dev what to do. You are _not_ asking the user what to do. This is akin of the git devs baking in a choice into git on what to keep in a merge conflict.

It's hard to trust you guys when you misrepresent like this. I thought long and hard on whether to respond confrontationally like this, but decided you really need to hear the push back on this.

[aboodman]

lol wut?

I represented that we ask the dev what to do:

> Zero asks your code what to do

You agree that's what we do:

> You are asking the dev what to do. You are _not_ asking the user what to do.

I get that your actual issue is you don't think that what we do is "the software equivalent of git asking the user what to do". But like, I also said what we do concretely in the same paragraph. It's not like I was trying to hide something. This is a metaphor for how to understand our approach to conflict resolution that works for most developers. Like all metaphors it is not perfect.

FWIW, there is nothing stopping a developer from having this function just save off a forked copy and ask the user what to do. Some developers do this.

Also FWIW, Zero does not allow offline writes specifically because we want to educate people how to properly handle conflicts before we do. I see down-thread this is the majority of your concern.

[Sammi]

I assumed you were doing offline support yeah. I've heard a lot about local first development lately, so I guessed this what what you guys are tackling too.

Without offline support AND you're doing real time updating of data, then conflict resolution is not a real world practical concern. Users will be looking at the same data at the same time anyways, so they generally see what data won out in case of a conflict, as they are looking at real time data as they are editing.

IF you had offline support, and for other sync engines that do: There is a real and meaningful difference between a backend dev and an end user of the application choosing what to do in case of a conflict. A backend dev cannot make a general case algorithm that knows that two end users want to keep or throw away in a conflict, because this is completely situational - users could be doing whatever. And if you push the conflict resolution to the end users, then you are asking a lot of those users. They need to be technically inclined and motivated people in order to take the time to understand and resolve the conflict. Like with git users.

[aboodman]

> Without offline support AND you're doing real time updating of data, then conflict resolution is not a real world practical concern.

I disagree with this. There are many real-world cases where keywise lww does the wrong thing. The article I linked up-thread covers many of them. Even a simple counter does the wrong thing.

This is where robust conflict resolution really matters in these systems, not the long-time offline case people often ask about.

You need robust conflict resolution to make correct software and maintain invariants in the face of write/write systems.

> A backend dev cannot make a general case algorithm that knows that two end users want to keep or throw away in a conflict, because this is completely situational - users could be doing whatever. And if you push the conflict resolution to the end users, then you are asking a lot of those users. They need to be technically inclined and motivated people in order to take the time to understand and resolve the conflict. Like with git users.

I agree completely. In my opinion the ideal offline-first write/write UI has never been built, but the team at Ink & Switch are closest:

https://www.inkandswitch.com/patchwork/notebook/

I think the perfect UX in many cases is that syncs goes ahead and tries to land the offline writes, but the user has a history UI where they can see what happened. Like how many collaborative apps do today.

But importantly in this UI the app would represent branches and merges. But unlike Git's fine grained branch/merge points, in this UI it would literally represent points where people went offline and made changes.

Users could then go back and recover the version of their data from when they were offline, or compare (probably manually in two tabs) the two different versions of the data and recover.

This does still ask users to compare and resolve conflicts in the worst case, but it is not a blocking operation or one that is final. The more common case is the user will go ahead with the merge and sometimes find some corruption. They can always go back and see what went wrong after the fact and fix. This seems like the right tradeoff to me of making the common case (no conflict) easy and automatic but making the uncommon but scary case at least not dangerous.

There also needs to be clear first-class UX telling users that they're going offline and what will happen when they come online.

I'm looking forward to someday working on this, but it's not what our users ask about most often so we're just disabling offline writes for now.

[probabletrain]

> Previous sync engines really want you to sync all data

Linear had to do all sorts of shenanigans to be able to sync all data, for orgs with lots of it – there's a talk on that here:

https://www.youtube.com/watch?v=Wo2m3jaJixU&t=1473s

[lifty]

Is Zero based on prolly trees?

[aboodman]

No. When we started the project we used prolly trees initially, but we don't need the content addressing feature for Zero and all the hashing was quite expensive. So now we just use a plain immutable b-tree:

https://github.com/rocicorp/mono/tree/main/packages/replicac...

[Jyaif]

> All this recent hype about sync engines and local first applications completely disregards conflict resolution

The main concern of sync engines is precisely the conflict resolution! Everything else is simple in comparison.

The good news is that under some circumstances it is possible to solve conflicts without user intervention. The simplest example is a counter that can only be incremented. More advanced data structures automatically solving conflicts exists, for example solving conflicts for strings exists, and those are good enough for a text editor.

I agree that there will be conflicts that are resolved in a way that yields non-sensical text, for example if there are 2 edits of the sentence "One cat":

One cat => Two cats

One cat => One dog

The resulting merge may be something like "Two cats dog". Something else (the user, an LLM...) will then have to fix it.

But that's totally OK, because in practice this will happen extremely rarely, only when the user would have been offline for a long time. That user will be happy to have been able to work offline, largely compensating the fact that they have to proof read the text again.

[SkiFire13]

This doesn't "solve" conflict resolution, it just picks one of the possible answers and then doesn't care whether it was the correct one or not.

It can be acceptable for some usecases, but not for others where you're still concerned about stuff that happens "extremely rately" and is not under your direct control.

> Something else (the user, an LLM...) will then have to fix it.

This assumes that user/llm knows the conflict was automatically solved and might need to be fixed, so the conflict is still there! You just made the manual part delayed and non-mandatory, but if you want correctness it will still have to be there.

[brulard]

> in practice this will happen extremely rarely, only when the user would have been offline for a long time.

I don't think it would happen "extremely rarely". Drops in connectivity happen a lot, especially on cellular connection and this can absolutely happen a lot for some applications. Especially when talking about "offline first" apps.

[Jyaif]

You have to use another device during that drop of connectivity on cellular connection, and edit the same content. That doesn't happen often.

[sgt]

> All this recent hype about sync engines and local first applications completely disregards conflict resolution.

Not really true though. I've used a couple of local sync engines, one internally built and another one which is both commercial and now open source called PowerSync[1]. Conflict resolution is definitely on the agenda, and a developer is definitely going to be mindful of conflicts when designing the application.

[1] https://www.powersync.com/

[Sammi]

My unfortunate point is that the dev cannot know what the user is doing, and so cannot in principle know what choice to make on behalf of the user in case of a conflict. This is not a code problem. It cannot be solved with code.

[sgt]

I've found that in almost all cases - the latest update "wins" strategy is fine. You could have two sessions working with conventional API calls and still have a conflict. As a dev you need to restrict what the user can do.

[porridgeraisin]

Precisely. The hype articles write all about the journey to The Wall, and then leave out the bit where you smash headfirst into it.

[lifty]

Very good point. The local-sync ecosystem is still in a young phase, and conflict resolution hasn't been tackled or solved yet. Most systems have a |last write wins" approach.

[jamil7]

> All this recent hype about sync engines and local first applications

Kind of but only really in the web world, it was the default on desktop for a long time and is pretty common on mobile.