Hacker News new | ask | show | jobs
by stevefan1999 1319 days ago
The very problem of SQLite: Single user only. Although SQLite does have WAL but it still doesn't allow you to do concurrent write unless you want to see file corruption.

This means SQLite is very much locked to things that works with one specific purpose and almost nothing else. Sure, you can be read-only, but you have to run alongside the app in the specific node, too.

Another problem (although without solving the single user mindset this wouldn't be a problem at all) is high availability. You want to make sure that your database won't get lost do you.

Things like Litestream [1] attempts to solve the SQLite backup problem it by continuously saving the database state and pack t up to S3-compatibles or file system but its just half the story. You want to make sure your operation not stopping. This is where HA comes in to save you from an emergency fixup when you are enjoying your holiday.

It doesn't mean that nobody tried to solve both these problems though. Ahem, introducing rqlite [2]. Although my own experience is not very great because the memory usage is quite high and does not fit my need (because the embedded device only has 512MB on it, and every byte counts, sorry), I guess that's the price to pay if you want to turn a non-multiuser, non-concurrently acccess database into one...Another honorable mention would be LiteFS [3] but I haven't used it yet so I have no say on it.

[1]: https://litestream.io/

[2]: https://github.com/rqlite/rqlite

[3]: https://github.com/superfly/litefs

3 comments

Many of us here writing web apps for enterprises can use SQLite with WAL with no issues.

Number of concurrent users range from tens and rarely hit hundreds. SQLite can handle that kind of traffic without any issues.

why would you do all this work when postgres sorted this all out over a decade ago?
Setting up Postgres is a PITA compared to SQLite. It comes bundled with python these days. Obviously it's going to be a trade off as to which one causes you more pain.
No docker access? postgres images are available in seconds.
So the solution is pulling down additional gigabytes of images and runtime to run the database?
Yes. It works great.
I manly work as a sysadmin for small companies. (Most people don't call us when they install new tools, they call when they exploded.) All my hatred goes out to the (Windows) programs (and its creators) that think they need MS SQL Express or the likes to save their two bytes of dust. All my love goes to the programs that just run/save from/to a UNC path.
postgres alpine is 90MB.

I can really recommend it, using a declarative docker-compose.yml file and then docker-compose command.

agreed, running pg locally is a pain. I use a cloud postgres instance (even for local dev). They're dirt cheap and it's not worth the hassle of working with a local pg.
What OS are you on? Running postgres on Linux is as trivial as it gets, and for Windows they have a nice installer. Or just use docker.
Does the docker have users set up? I seem to remember that being a pain point in the initial setup.
I love Postgres.app on mac, makes running locally a breeze. Running Postgres fast on the cloud is what I always struggled with (without paying enormous sums)
>> agreed, running pg locally is a pain

If you have a single process accessing the database, use SQLite. If not, use Postgres.

which cloud do you use for local dev?
aws, gcp, digital ocean and supabase all have pretty good free tiers
There is also Cloudflare D1, which is now in public alpha.