Hacker News new | ask | show | jobs
by villasv 1701 days ago
Introducing Elasticsearch to your stack increases complexity tenfold. Elasticsearch itself is a complex beast and if it's your first distributed cluster, beware the time spent operating it ^1. Apart from that, you now have to manage the data replication between your database (source of truth) and Elasticsearch (search optimized view). This can be easy or nightmarish, depending on the nature of your application.

So the first thing I recommend anyone contemplating this decision is: can Postgres FTS implement what you need? Then prefer staying inside Postgres. The scenario I'd feel justified in adopting Elasticsearch are: immense amounts of data (at least above a million of documents), complex user queries (e.g. boolean clauses and field specification), complex NLP and relevance engineering.

My opinion is the same for Apache Solr. There are other search options (Typesense, MeiliSearch, Vespa) that I never tried and can't compare.

1. Yes even managed Elasticsearch offerings like AWS OpenSearch and Elastic Cloud require quite a bit of operations. Sharding, index configuration, node sizing, disk monitoring, threadpool monitoring, allocation issues... frequent worries that managed offerings don't handle for you.

3 comments

I love the “search optimized view” framing. That is indeed the case for us. It’s hard to store objects inside postgres with loose schema (eg not all attributes for the document are always present).

I keep dreaming that AWS will someday release the unholy child of dynamodb and solr! Maintenance of ES can wear you down.

I have never tried AWS managed elasticsearch, but Azure Search doesn't have many knobs. Most of the time it's just instance size, replica and partition count.

The downside is that the service is probably called "Azure Cognitive DeepMind Quantum DevOps Search" by now.

We use Elasticsearch and Kibana to store, search and visualize tranzactions, microservice calls and logs. We don't replicate anything from databases to Elasticsearch.

I was just curious of the performance delta, if any.