Hacker News new | ask | show | jobs
by igama 2312 days ago
One way for example would be store all events you want to write to ElasticSearch to a Queue (like rabbit, Kafka, etc) then have a set of workers/importers (this can even be logstash with input from queue and output ES) that read the events from that queue and store them in ES.

This way your system becomes a sync, and your system above is not affected if ES becomes slow writing or goes down for a reason. Your events will remain stored in the queue waiting to be processed.