[bshipp]

I understand everyone wants concurrency because that's the way everything else in programming works these days, but SQlite in memory or on an SSD (in WAL mode) writes so fast that it makes infinitely more sense to dedicate a single database "write" worker that deals with the dB itself while allowing multiple "read" workers to access the database concurrently. Other workers are spawned to process their data and shovel their writes into a queue to feed that single worker.

In my experience, the CPU utilization of process workers preparing data into SQL statements is generally the chokepoint in most/script programs instead of SQlite write speeds themselves. Of course, this is dependent on numerous factors such as the existence of indices, compexity of write operation, etc., but--as a rule--SQlite is much more efficient if I dedicate one process to handle all writes and optimize that process for speed (i.e. batch writes, etc.).

[tracker1]

I agree... My thinking is that if you're growing beyond a single process accessing the database, either put a service on top of the SQLite db, or move to a db more suited to the needs/workflow.

I generally think in terms of SQLite -> Firebird (maybe) -> PostgreSQL -> CockroachDB (or others). Just depends.

[Scarbutt]

SQlite is much more efficient if I dedicate one process to handle all writes and optimize that process for speed (i.e. batch writes, etc.).

That increases complexity on the application side though.