Hacker News new | ask | show | jobs
by Blikkentrekker 1663 days ago
locate is, for whatever reason, tragically slow. The database format it uses is nonsensical and completely optimized for size on very outdated assumptions.

I use an implementation I have written in the shell itself whose database format is nothing more than every file path on the system separated by null bytes, that is simply grepped to find files; the speed difference is absurd.

  —— — time locate */meme.png
 /storage/home/user/pictures/macro/meme.png

 real  0m0.885s
 user  0m0.806s
 sys 0m0.010s
  —— — time greplocate /meme.png$
 /storage/home/user/pictures/macro/meme.png

 real  0m0.089s
 user  0m0.079s
 sys 0m0.011s
This implementation is highly naïve and simplistic, and offloads all the searching to GNU Grep, yet outperforms the actual `locate` command by an order of magnitude.
3 comments

And plocate is yet orders of magnitude faster than GNU grep. :-) And updates its database faster. You don't specify which locate you're using, but mlocate and BSD locate are basically obsolete by now.

(Disclosure: I'm the author of plocate.)

I'm a huge fan and use your program basically daily. I don't know why it's not the default in most systems.
Thank you! It's becoming the default now, slowly (e.g., it will be the default in Debian and Ubuntu from the next releases, and Fedora is in a process to make it replace mlocate right now). It's just a tad too new, only about a year since 1.0.0. :-)
Nice. I do something similar. Why null byte instead of new line?
Presumably to support filenames with newlines in them.
Yeah but… gee
Why take the risk? I'm fairly certain I have no files with newlines on my system but I see no reason to take the risk.
Unrelated, but can I ask about your prompt?
They are simply em-dashes, if an error should occur, they are replaced with numbers to indicate the error code; they are also color-coded to indicate when I'm not for instance in an SSH session by changing to another color:

  —— — true
  —— — false
  —— 1 sh -c 'exit 120'
   120 sh -c 'exit 20'
  — 20

It doesn't show the colors here of course.