[timanglade]

Hi, former Apache CouchDB guy, former Cloudant employee, and former Realm employee here… I (still) love CouchDB but at the end of the day I think its sync is a better fit for server-to-server or web client-to-server use-cases, not mobile-to-server.

There’s tons of stuff CouchDB sync doesn’t support that end up being a huge problem with mobile apps… Here are the top 3 things Realm adds in my opinion: great client-side performance, native models that are extremely easy to use, and conflict-free sync (not conflict resolution!). With my own eyes I’ve seen a generation of developers try CouchDB for mobile apps and then abandon it because of the limitation of its approach. Personally I’m glad to see someone try to bring more options to developers. I think this is the first general-case true sync (à la Google Docs) for mobile, and I hope it will continue to spur more iterations & innovation in other open-source projects as well.

[ramses0]

Can you explain this statement: "There’s tons of stuff CouchDB sync doesn’t support that end up being a huge problem with mobile apps"

You lay it out there and then talk about what Realm adds, but never what Couch lacks. And are you including PouchDB in your evaluation of CouchDB for "Client-Side Performance"?

As I understand PouchDB is performant, uses JSON (pretty much the definition of "Native Models"), and allows server-arbitrated conflict resolution with stored revisions, making the simplest conflict resolution "last-in-wins" with user UI for "undo" (or any JS-object merge library).

Thanks!

[andkon]

I think you're misunderstanding what "Native Models" means in the context of iOS work. Saying that a database uses JSON immediately, to many ios devs, recalls years of writing very fragile json serialization code. Looking through CouchDB's iOS example apps confirms that yes, basically everything that isn't a string or int has to be serialized into JSON. That's a pretty enormous annoyance, and incorporates a lot of additional cognitive overhead when dealing with data that Realm simply handles properly for iOS from the beginning.

I'm deeply relieved to see that they're launching something that does something about the fragile ios architectures built around JSON. It's similar to what they did with their mobile database: they actually listened to what mobile devs found annoying, and then they built something that completely sidestepped all the fragility problems that emerge when you wrap sqlite with an ORM. Instead, they made an approach that fit within our workflow, as opposed to taking us further from it.

[ramses0]

Ahhh, got it. When I hear "offline-first" my mind's been in webdev-mobile, not appstore-mobile. Thanks for clarifying!

[janl]

CouchDB doesn't have any iOS code, are you perhaps referring to Couchbase?

[andkon]

Ohhh my god, you're totally right. I was looking at this: https://github.com/couchbase/couchbase-lite-ios

[dismantlethesun]

The things Realm adds are what CouchDB lacks. Those things are essentials from a certain point of view.

[avtar]

There’s tons of stuff CouchDB sync doesn’t support that end up being a huge problem with mobile apps… Here are the top 3 things Realm adds in my opinion: great client-side performance, native models that are extremely easy to use, and conflict-free sync (not conflict resolution!).

Thanks for listing those potential benefits. Any idea if those CouchDB issues are covered in detail somewhere?

[daleharvey]

I work on PouchDB, in terms of those points:

"great client-side performance": PouchDB is more than fast enough for most use cases, its just a wrapper over indexedDB. (there are areas we can improve, particularly performance of map reduce / mango queries).

"native models that are extremely easy to use": Not sure what that means, PouchDB and CouchDB use json, its native and easy to use in JavaScript

"conflict-free sync (not conflict resolution!)": Certainly going to have to take a look at this, any conflict free sync protocol I have seen put a lot of limitations on how data within your application is structured but conflict resolution certainly is something that can be improved within Pouch / Couch.

I do think there is a bunch we can (and are) improving in PouchDB and its great to see alternatives, will be looking into this further now. But I certainly do not agree that (C|P)ouchDB are not suited for mobile <-> web sync, thats the primary use case I work on Pouch for.

[nolanl]

(I also work on PouchDB.)

For "conflict-free sync," according to the docs, this appears to be last-write-wins along with a special case for deletes: https://realm.io/docs/realm-object-server/#conflict-resoluti... . (Incidentally Couch/Pouch also has a special case for deletes, but it does the opposite behavior by default – deletions lose to updates.)

There's also some language in here about how "insertions in lists are ordered by time;" I'd be curious to know how they coordinate clock times across nodes. Ditto for "last update wins" – important to know how "last" is determined.

In general, I tend to be skeptical of systems that hinge on last-write-wins, but LWW is definitely a model that works in a lot of cases where you're not too concerned about the possibility of losing data, and it's also a simpler mental model for the app developer.

[astigsen]

Realms approach to conflict resolution is very different from what you see in products like Couch. Realm is an _object_ database, so in contrast to document databases where conflict resolution happens very coarse grained at the document level, in Realm it happens at the object level (and actually all the way down into individual properties).

This essentially means that the entire object graph is treated as one big CRDT [0], transparently handling conflict across individual objects, properties and relations.

When talking about ordering by time, both in the context of LWW and insertions in lists, we are talking about vector clock time, not necessarily device time (or though that is taken into account as well).

[0] https://en.wikipedia.org/wiki/Conflict-free_replicated_data_...

[gritzko]

An-object-graph-as-a-CRDT is a really interesting concept. So far, I am not aware of any literature formalizing/studying it. I can only recall the recent CRDT JSON paper by Kleppmann et al, which is significantly less ambitious. Is your approach documented?

While reading the announce, my first guess was some recursive per-field LWW, but you mentioned vector clocks. That is intriguing.

[pfraze]

LWW is a tradeoff. You'll be better off selling it as an automated-resolution option on top of manual resolution, instead of an innovation over manual resolution. (MV registers are a CRDT as well!)

[relics443]

"native models that are extremely easy to use"

This means native code (specifically not JSON). So in Java you would be dealing with POJOs, etc...

[WA]

A bit OT, but I have to compliment the person who wrote the docs[1] for PouchDB. They're quite entertaining and easy to understand. Well done!

[1]: https://pouchdb.com/guides/

[nolanl]

Thanks, I'm that person. :) I tried to write it as a "missing manual" for Couch/Pouch, and had a lot of fun in the process! It needs some updates for newer stuff, though: pouchdb-find, custom builds, Chrome DevTools storage UI improvements, etc.

[janl]

How is "conflict-free sync" done? The usual approach is last-write-wins, which is just a euphemism for "randomly losing user data", or CRDTs or Operational Transforms, which severely limit the data models one can sync.

I'm happy to learn about a silver bullet here.

[bigfish24]

It is true that most Operational Transform systems have limited data model support, but that's an area Realm is quite unique on. The Realm Mobile Platform supports all of Realm database's current data types, such as native support for links and ordered lists. Also have plans for much more including: substring changes, counters and sets.

[janl]

Thanks, that's gonna be the limiting factor then, developers not finding a way to represent some of their storage needs on this platform.

It's a fair trade-off, but hardly neither objectively worse or better than couch-sync. I'm excited to see where Realm goes.

In my very unscientific approximation couch/git-style sync solves about 80% of general use cases, CRDT/OT cover another 10-15% and the rest are poor souls that have to do something custom. (All this excluding binary sync/rsync/BitTorrent/Dropbox.)

[striking]

CRDT, but also vector clocks.

https://news.ycombinator.com/item?id=12590753

[gritzko]

Some CRDTs employ vector clocks. CRDT is a veeeeery broad family of algorithms and data structures.

[striking]

Is that not what I stated (even if it was poorly phrased, and I can no longer edit it)?

[janl]

Oh, and while there is still room for improvement, we've improved the server-mobile sync case by an order of big-O-magnitude. And this is a first class design target for the future.