Hacker News new | ask | show | jobs
by ademup 2613 days ago
I'm curious if this scales down well. The test was done on "One AWS c4.8xlarge with 36vCPU and 60GiB Memory". But could I run this on a tiny vps to index, search, catalog my million-odd documents?
3 comments

SQLite's FTS5 module works surprisingly well on that number of documents, and stores the search index on disk instead of RAM.
You can check out Sonic[1] and Tantivy[2] both are lightweight search engine

[1] https://github.com/valeriansaliou/sonic

[2] https://github.com/tantivy-search/tantivy

I'd guess redis performs _better_ in that case since there minimum overhead for redis is much lower than elasticsearch.
Wouldn't Redis need to keep the whole dataset in memory?
In my experience to get decent "show off in benchmarks" performance with ES you want your index to fit in RAM as well, but I haven't done much work with ES outside of high RAM boxes.