[sph]

Are there narrowly focused tools in the observability space even?

I was shopping for one after being outside of this field for a while, and they all do the 101 features and the kitchen sink model, which adds onto the complexity. DataDog, Grafana, but also the open source ones like SigNoz itself.

Ages ago it was all about metrics, today it's metrics traces logs APM alerting exceptions and a dozen other acronyms, on top of the protocols (statsd, Prometheus, OpenTelemetry), paired with crazy complicated yet unwieldy graph building UIs. Let's not even talk about pricing models. The entire business model is based around having one more checkmark in the feature list than the competition. The wire format (OpenTelemetry) has never been the pain point in this space.

For a moment, I seriously considered just going back to the 2000s and using RRDtool.

[phillipcarter]

Most new observability tools start narrow but every economic incentive is to expand. Which makes sense, really, because most production systems people have are complicated as all hell and have tons of different needs. Some tools are better than others at containing the chaos -- I will humbly submit that the one I work for, Honeycomb, is one of the best at doing this -- but support for several telemetry signals, visualization tools, alerting systems, dashboarding systems, etc. are all what people eventually ask for as they roll out observability to more of their production systems.

Put differently, when you have sufficient observability of your entire system, you now have a complete abstraction of that system represented in some other UI and data streams. There's just no way out of the fact that for larger systems, this will be complicated, and the tools that can represent this reality must also be complex.