Hacker News new | ask | show | jobs
by linsomniac 1578 days ago
Years ago this was very similar to our use case, only much smaller file sizes, and Reiser saved an amazing amount of overhead.

The systems were hosting map tiles of the entire Earth, and a lot of them were solid blue (ocean) or solid beige (unoccupied land). Those files were something like 60-300 bytes, going from memory. Reiser was basically the only FS that would handle that reasonably.

1 comments

Oh I have a repository of tens of millions of small json files, and xfs chokes on that specific partition it's amazing. 5 minutes for a ls in the root folder (with only 2 folders). Tried moving it disk to disk in case I was on a broken disk, but no.
Unless there is a good reason, I think in that situation I would probably try to avoid storing those tens of millions of small json files directly on the filesystem. For example : have you considered storing that data in jsonb/postgresql https://www.compose.com/articles/faster-operations-with-the-... ? (or even just in an sqlite db https://www.sqlite.org/fasterthanfs.html )