[dozy]

To address the speed problem you could still leverage sqlite locally as cache layer in between your in-memory cache and longer-term google spreadsheet cache, and use a repeating job to move the sqlite data over to google spreadsheets (and subsequently delete those rows in the sqlite, keeping the sqlite db at a small size).

[hakanu]

that's a good idea actually. It will be safer for retries thanks! For future reference, increasing the number of rows persisted makes the time complexity also increase.