[throwawaw]

> Importantly for us, Roam differs from other webapps in that it doesn't store all state and history in its backend. Instead, Roam's backend just stores a snapshotted Datascript database (updated ~daily as far as I can tell) and the list of transactions since that last snapshot.

If I'm reading this right... A) OP was hosed if they didn't finish this within ~24 hours. That's fun. B) I now know why Roam takes a long time to load -- it's replaying every edit I made since its last snapshot!

Edit:

> When a message is more than 16KB, it's split into multiple messages without wrapping - so we'll need to stitch these bigger messages together. One way to detect a non-split-message is to just try and parse it as JSON - if it's valid, we can say it's non-split. (There's an edge case we're unlikely to hit here: if the 16KB chunk just so happens to be valid JSON as well we'll be out of luck. Lucky for us, I didn't run into this!)

This is an extremely inconsequential detail to focus on, but I don't see the edge here at all. What's an example 16KB string that OP's method would actually fail to parse? (Maybe I'm failing to grok what "without wrapping" means here.)

[cwackerfuss]

Isn't it impossible for the 16kb partial to be valid json? Json starts in either a curly or square brace -- valid json more than 16kb would never close the outermost brace in the first 16kb of data, otherwise it would be invalid