[tonyplee]

Thx, smarx

Good info!

10MB database limit is too small - somehow I would image dropbox wants app to create hundreds of MB if not GB of data. :-)

Are there any test code available? Open Source / on github?

It is much easier to understand the flow from the test code than API doc. If test code is available, it is easily to morph that info benchmark type code.

[smarx]

The Python library is open source, but no, I don't think we have any open source test code.

Keep in mind that file data should go outside of datastores (in files), so the 10MB is for the per-user, per-app structured data. E.g. contacts, app settings, game state, etc. 10MB actually goes a pretty long way there.

Also keep in mind that, at present, all of the SDKs load the entire datastore into memory, so there's a pretty low limit to how big you want a datastore to get. 10MB is a comfortable limit for now.

[tonyplee]

Thx again for the info.

I see why the 10MB limit.

What's the Pro/Con of this compare to just use sqlite on a dropbox volume?

From what I read, the API doesn't do any operational transformation such as merge, delta the changes' etc. The client app is expected to do it.

[smarx]

The main pro is the automatic merging. "The API" is a fuzzy term here. There's an interaction between a client and a server, and the client is running an SDK. In the case of the Datastore API, the server doesn't perform any merging or OT, but the client SDK does. Your code just gets conflict resolution for free. The Python SDK is a bit of a special case in that it doesn't implement this logic, but the others (JavaScript, iOS, Android) do.

In contrast, if you make a change to a sqlite database on two different devices, you now have two different files and no way to merge them. (There are people who have used Dropbox to sync sqlite databases this way, and they've ended up writing diff/patch over sqlite to merge changes. Using datastores is a lot simpler and more likely to be correct.)

[tonyplee]

Hmmmmm, my understanding of OT is that you have to implement that on Server.....

Only the server has understanding of the most current states of the dataset and can try to sync up with multiple client at the same time.

[smarx]

This is incorrect. OT can indeed be implemented on the client.

The way it works in the Datastore API is that the client sends its changes to the server with an attached "parent revision." If that parent revision is the revision that the server has, then the change goes through (no conflict). If the revision doesn't match, then the change is rejected by the server, and it's up to the client to pull down the latest changes from the server, merge things (via, in the case of lists, OT), and then try again.