[herf]

I think Takeout is a heroic effort, but it's pretty clear most of these datasets would be much more useful in "streaming" form, like you would use for a database migration. For instance, you almost always would want to trial a new service for a few months before "moving" to it entirely, and you might run two things like this in parallel for a long time.

If you want to download a snapshot every so often, you should be able to give it a token representing the last timestamp, and have it give you all the things that have been added since then - most datastores can do this. Then, if the service supports it, it should also indicate changes to old data (including deletes) - but this requires some additional state so it probably wouldn't work for every case.