[alexkoeh]

This is awesome. As our userbase grows I've been wondering how to best address this problem.

When we have a migration that requires more than 500 batch writes, we first got around that by building an array of batches with 500 writes per entry, then committing all of it at the end. Then we discovered the bulkWriter API in the node.js library which gets around the 500 batch limit. It looks like `createBatchMigrator` only supports up to 500 writes due to the firestore batch limit. Is that correct?

[anarkafkas]

Thank you! Yes, that's correct. The limit for a batch migrator must be 500 because it uses Firestore write batches internally.

I'm currently writing another migrator that won't be using Firestore batches, it'll just use the good old Promise.all(). I'm planning to add more capabilities soon like error-resilient traversers using different traversal strategies, the ability to re-traverse the docs that couldn't be migrated the first time etc.

[nicoburns]

It looks like this will support more than 500 by calling the batch API repeatedly in a loop (the bulkWriter API is likely doing similar).

If you really want to address the issue of a growing userbase I'd highly recommend moving off of Firestore as soon as possible. It's really very inefficient at things like bulk updates (e.g. whereas in a SQL database you could use an UPDATE WHERE, in Direstore this is impossible without first reading every document than writing it back)