Hacker News new | ask | show | jobs
by edmundsauto 929 days ago
So you store the actual image data in redis? That's interesting, no issues with storing binary data?

I ask because I've always been taught to not store files in a database. This use case sounds like an interesting exception.

1 comments

Files are just a bunch of bytes. No harm in putting them in a database.

There were some benchmarks, I couldn’t fine where SQLite was faster than native file system at retrieving, searching and adding files to a large directory.

SQLite reads and writes small blobs (for example, thumbnail images) 35% faster¹ than the same blobs can be read from or written to individual files on disk using fread() or fwrite().

Furthermore, a single SQLite database holding 10-kilobyte blobs uses about 20% less disk space than storing the blobs in individual files.

The performance difference arises (we believe) because when working from an SQLite database, the open() and close() system calls are invoked only once, whereas open() and close() are invoked once for each blob when using blobs stored in individual files. It appears that the overhead of calling open() and close() is greater than the overhead of using the database. The size reduction arises from the fact that individual files are padded out to the next multiple of the filesystem block size, whereas the blobs are packed more tightly into an SQLite database.

https://www.sqlite.org/fasterthanfs.html