Hacker News new | ask | show | jobs
by NorwegianDude 143 days ago
I do t have time to test myself now, but it would be interesting to see a proper benchmark. We all know it's not suitable for high write concurrency, but SQLite should be a very good amount faster for reads because of the lack of overhead. But how much faster is it really?
3 comments
adzm 142 days ago
as an in memory database, I got around 40,000,000 reads per second. Using WAL and a file rather than in memory, around 900,000 reads per second. This is single threaded, for a simple integer ID to string Value query, and a few years old at this point, and only minor config optimizations eg not even using memory mapped io and a ~3gb database with a million or so rows on a Windows machine. The performance really is amazing.
link
NorwegianDude 142 days ago
40 million reads per second, on a single core? 40 million reads/s is 25 ns per read, that is faster than any RAM I know of.
link
yencabulator 141 days ago
It's not like every read would make a separate trip all the way to RAM, caches are a thing and SIMD pipelines/parallelizes comparisons within a hash bucket quite well. Lookups from a hash map should amortize to something like 5-20ns per lookup these days. Abseil's Swiss Tables for C++ and Rust's Hashbrown both should reach that.
link
NorwegianDude 141 days ago
If you're looking up values from a 3 GB DB, most would have to hit RAM. Lookups form a hash map can be fast, but SQLite does quite a bit more than just a hash map lookup, and it would usually hit RAM, not L3 cache.
link
yencabulator 141 days ago
Parent comment said "with a million or so rows". I looked up numbers for benchmarks with ~1M entries in the hashmap.

1M 64-bit integers is only 8MB, that's still a small keyspace.

link
adzm 140 days ago
Perhaps relevant is that probably only 25% of the IDs that get passed to the select actually have values and this is using one thread for the benchmark. It's too convoluted to share, but indeed when using the in memory database on a lower spec laptop currently I still get up to 20-30m reads per second, pretty close to the 40m on the big box.
link
jacobobryant 142 days ago
In some informal benchmarks I wrote using queries + data from a web app I develop, sqlite queries were about 5x faster than postgres.
link
solumunus 142 days ago
Orders of magnitude I would imagine. Very significantly faster.
link