[bob1029]

I am glad to see more projects down these paths.

We've been sticking JSON documents into SQLite databases for years, simply because we can't be bothered to map thousands of business facts to individual columns. Turns out that this approach is fast enough that our customers can't tell the difference either way.

Decisions like this can make for one of those 10-100x speedups in development timelines. Even if you are convinced that JSON serialization+SQLite are too slow, you should still try it to make sure. You will almost certainly be surprised.

[sethammons]

Love it. Got some questions: What request volume does it perform against? Do you shard requests? How large is the dataset? What is an acceptable response time? Any major slowdowns at the highest percentiles?

[bob1029]

Our request volume has a strict upper bound that we can depend on. The part of the user base touching JSON+SQL code paths is well-constrained.

For the most contentious area of our application - updating business state instance per user - We find that we are able to serve on the order of 1~10k requests per second. The size of these datasets is around 0.5~15 megabytes.

There isn't a whole lot of other magic involved. WAL is the most important thing for improving throughput.