[otisg]

> so tired of Elastic just randomly killing their clusters out of nowhere and having to spend basically triple to deal with it that they're bringing it all in house.

That is because Elastic Cloud is not a fully managed Elasticsearch. People often don't get that with Elastic Cloud you are still responsible for your ES cluster. That's one of the differentiators that e.g. Sematext has (disclaimer: founder).

> AWS's hosted solution is poor, Logz.io and ElasticCloud are expensive.

Right. Have a look at https://sematext.com/logsene pricing. People say Sematext compares favorably to Logz.io and Elastic Cloud.

[sandGorgon]

Why are all ELK stack based logging solutions so much more expensive than custom rolled solutions like logdna, datadog, papertrail,etc.

The per gb cost on the other ones start at 1.20 $/gb and goes till 2$/gb. While almost all hosted ELK solutions start off at 3$/gb.

Im asking because i would very much like to adopt an ELK based hosted solution..but I'm not able to justify paying double. Is it that running+resource costs for ELK are so high that the extra charge needs to happen ?

[manigandham]

Yes it's mostly about resource costs. ES is a generic search that can handle logs but isn't focused on it, and indexing everything can be costly. There are major efficiencies gained in creating your own log-focused storage and access methods with object storage, columnar formats, and zone mapping, etc.

[busterarm]

Datadog, etc are not cheap when you're paying per agent/per month to ship your logs in the first place. Which they want you to do, of course.

[sandGorgon]

The pricing is not per agent per month. The pricing for everyone is always either per million events or per gb.

You do not need an agent. You can ship directly from syslog ( https://docs.datadoghq.com/integrations/rsyslog/?tab=datadog...)