[iheart2code]

Very cool document. I kind of got stuck at delta queries, though. How do you implement that? I can't find any reference to delta/removed queries on Mongo, Postgres, or MySQL. Do you just keep all records in the database and add a "removed" field? How would that solution work with user privacy & users truly wanting to remove data?

[dsp1234]

Delta queries can be serviced by any system that supports temporal tables[0][1], or another mechanism where data changes are tracked.

[0] - https://en.wikipedia.org/wiki/SQL:2011 (temporal support)

[1] - https://msdn.microsoft.com/en-us/library/dn935015.aspx

[2] - http://clarkdave.net/2015/02/historical-records-with-postgre...

[ChrisAtWork]

Delta queries are pretty easy, assuming you have a rich data store behind your data.

As others have said, your data store needs to be able to say, "Show me changes since XYZ". Most of the Big Apps can do that, and from there the problems is one of API Semantics.

This doc addresses the API Semantics, rather than the application design. To try to solve the design problem would be impossible as every application is different.

[iheart2code]

I understand what the doc is trying to convey, I just hadn't ever thought of the application-level logic for record deltas before, so I got incredibly distracted from the document. It made me wonder how other people, not using temporal tables, are accomplishing this right now.

[ChrisAtWork]

I don't think I've ever seen delta's done at-scale with actual temporal tables, but rather with transaction Id's of some sort.

For example (based on the doc) when you register a delta link, there's a row written to Azure Tables saying, "owner:iheart2code, deltaToken:foo, lastRecordIdSeen:123".

When you then make the Delta request, we look up "foo" for you, find that the last id you've seen is 123, and then only give you records from the transaction table with an id larger than that.

Temporal is always a can of worms, as clocks are impossible to keep in sync and there are endless race conditions.

Making the delta tokens causal, rather than temporal, is the way to go. Anything else is brutal in the distributed systems world...