[thecodrr]

Ah war stories.

Just a few weeks back a user came to us after failing to migrate GBs of their data off of Evernote. This, of course, included attachments. They had successfully imported & synced 80K items, but when they tried to login on their iPhone, the sync was really, really slow. They had to wait 5 hours just to get the count up to 20K items. And that's when the app crashed resetting the whole sync progress to 0.

In short, we had not considered someone syncing 80K items. To be clear, 80K is not a lot of items even for a local-first sync system, but you do have to optimize for it. The solution consisted of extensively utilizing batching & parallelization on both the backend & the users' device.

The result? Now their 80K items sync within 2 minutes.

[raspasov]

In this case, would the optimizations/fix be easier if you were using traditional client-server setup vs local-first?

[yencabulator]

The problem wouldn't exist. This was about the phone fetching 80k new items from the server. If the phone just shows the item you're looking at, one at a time, and doesn't try to sync everything, there's no such problem.

[raspasov]

There's no restriction inherent to CRDTs/local-first around a partial sync. You are not required to sync everything.