Hacker News new | ask | show | jobs
by smarx 4617 days ago
The Python library is open source, but no, I don't think we have any open source test code.

Keep in mind that file data should go outside of datastores (in files), so the 10MB is for the per-user, per-app structured data. E.g. contacts, app settings, game state, etc. 10MB actually goes a pretty long way there.

Also keep in mind that, at present, all of the SDKs load the entire datastore into memory, so there's a pretty low limit to how big you want a datastore to get. 10MB is a comfortable limit for now.
1 comments
tonyplee 4617 days ago
Thx again for the info.

I see why the 10MB limit.

What's the Pro/Con of this compare to just use sqlite on a dropbox volume?

From what I read, the API doesn't do any operational transformation such as merge, delta the changes' etc. The client app is expected to do it.

link
smarx 4617 days ago
The main pro is the automatic merging. "The API" is a fuzzy term here. There's an interaction between a client and a server, and the client is running an SDK. In the case of the Datastore API, the server doesn't perform any merging or OT, but the client SDK does. Your code just gets conflict resolution for free. The Python SDK is a bit of a special case in that it doesn't implement this logic, but the others (JavaScript, iOS, Android) do.

In contrast, if you make a change to a sqlite database on two different devices, you now have two different files and no way to merge them. (There are people who have used Dropbox to sync sqlite databases this way, and they've ended up writing diff/patch over sqlite to merge changes. Using datastores is a lot simpler and more likely to be correct.)

link
tonyplee 4616 days ago
Hmmmmm, my understanding of OT is that you have to implement that on Server.....

Only the server has understanding of the most current states of the dataset and can try to sync up with multiple client at the same time.

link
smarx 4616 days ago
This is incorrect. OT can indeed be implemented on the client.

The way it works in the Datastore API is that the client sends its changes to the server with an attached "parent revision." If that parent revision is the revision that the server has, then the change goes through (no conflict). If the revision doesn't match, then the change is rejected by the server, and it's up to the client to pull down the latest changes from the server, merge things (via, in the case of lists, OT), and then try again.

link