Hacker News new | ask | show | jobs
by waitwaitwhay 4114 days ago
According to their documentation, the data storage for log events is Elasticsearch and if the Elasticsearch data is lost then the logs are gone. Consideribg that ES is not a database and may lose data this sounds a bit scary to me.
3 comments
_up 4114 days ago
My understanding is that appending only is fine. So if your use case is logging elasticsearch is totatly fine. You probably can't compare it to the robustness of MySQL & PostreSQL. But most NoSQL are not known to be that robust either.
link
toomuchtodo 4114 days ago
You can solve this in one of two ways:

1. Whatever mechanisim you're using to send data to Graylog, you send that data to S3 as well. You can then reload Graylog at anytime with S3 data.

2. Backup your Elasticsearch nodes to S3

I should've mentioned I run this in AWS. Sorry about that!

link
pswenson 4114 days ago
read up here... https://www.elastic.co/blog/resiliency-elasticsearch/

they had a major data corruption bug last year but have taken measures to correct it. I asked if elasticsearch could be a source of truth at elasticon and they didn't say yes, but they indicated that you "could do it" and it is a goal

link