[ec109685]

Sad. Splunk should be more fantastic. They have done the heavy lifting of taking streams of data at high volume, which should be the basis to build a log search product, metrics And alerting, and observability.

Instead, each of these systems have their own collectors and correlating from one to the other is hard. A canonical log line is so much more valuable than a metric collected every 60 seconds, and the former can derive the latter: https://stripe.com/blog/canonical-log-lines

Splunk should have been the lynchpin.

[flounder3]

Absolutely agree. It’s tragic.

Shoving SignalFx down our mouths and trying to get us to create “metric” indices was the straw that broke my back.

[jcims]

I built a PCI compliance solution for a customer back in 2008 for ~$200k all-in when the closest competitor's bid was five times that. The product was amazing at runtime but of course had some idiosyncrasies in how it was configured and whatnot. I've been a user (only) of Splunk heavily ever since and just last year got pulled into a project to migrate a huge install to a cloud platform. It felt like I got into a time machine...there were seemingly zero administrative or architectural improvements to make the product more manageable or supportable in the 10+ years since I had last looked at it from an ops perspective.

I'm sure that's not 100% true but it felt like it. Trying to build Splunk on top of a modern IaC deployment methodology is a huuuuge lift.

[ec109685]

Yeah, team decided K8s for Splunk would be too much work and ended up needing to use vanilla VM’s with block storage on an open stack env on prem.

Pretty lame not cloud native.