Hacker News new | ask | show | jobs
by tootie 4085 days ago
How does SQLite handle replication? Can I have 10 app server nodes reading from one SQLite DB? NFS?
4 comments

From the article: Situations Where A Client/Server RDBMS May Work Better... High Concurrency: SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time.

I've used SQLite over NFS for replication. It worked fine.

The database is just a file, so just copy it (rsync works great for compressing this process over a slower network). If you need to replicate without blocking writes, you can do that with some scripting around filesystem (e.g. btrfs) or block layer (LVM) snapshots.

Though again, if you want true streamed (i.e. per query) replication, features like hot standby, etc... that's not part of the model and you're better served by a server-model RDBMS.

SQLite has a backup api, so not entirely what you might need - but you can get a consistent snapshot without affecting much the db using the backup API:

https://www.sqlite.org/backup.html

The way I've used it (for a db of 300-400mb loaded in-memory in a 64-bit app) was to load the .db in memory, then detect if there are changes outside (notified, or updated the .db file timestamp from a different app using the db), then load a second copy of the .db and compare, then build diffs and report them in some kind of API fashion.

It was used for live update of various level editing data for a game level editor. Possibly not the best fit (too much hierarchical data), but worked nonetheless.

One thing that I miss dearly from SQLite is some form of postgres arrays, but I'm so glad that recursive with has been added recently to SQLite which allows for hierarchical data (child/parent storing ids to each other) to fetch without some extra information.

I'm not sure if SQLite writes over NFS work, I wouldn't trust it. Previously I have done SQLite replication by simply rsyncing the database to application servers at certain times.
Writes definitely work, but concurrent writes will hose the database. SQLite relies on filesystem locking (which NFS does not handle properly) to handle write concurrency.