Hacker News new | ask | show | jobs
by maccard 947 days ago
I've spent a small amount of time in datadog, lots in grafana, and somewhere in between in honeycomb. Out applications are designed to emit traces, and comparing honeycomb with tracing to a traditional app with metrics and logs, I would choose tracing every time.

It annoys me that logs are overlooked in honeycomb, (and metrics are... fine). But, given the choice between a single pane of glass in grafana or having to do logs (and metrics sometimes) in cloudwatch but spending 95% of my time in honeycomb - I'd pick honeycomb every time

4 comments

Agreed - honeycomb has been a boon, however some improvements to metric displays and the ability to set the default "board" used in the home page would be very welcome. Also would be pretty happy if there was a way to drop events on the honeycomb side for a way to dynamically filter - e.g. "don't even bother storing this trace if it has a http.status_code < 400". This is surprisingly painful to implement on the application side (at least in rust).

Hopefully someone that works there is reading this.

It sounds like you should look into their tail-sampling Refinery tool https://docs.honeycomb.io/manage-data-volume/refinery/
Yep, this is the one to use. Refinery handles exactly this scenario (and more).
I think Honeycomb is perfect for one kind of user, who's entirely concerned with traces and very long retention. For a more general OpenTelemetry-native solution, check out Signoz.
Have you tried the traces in grafana/tempo yet? https://grafana.com/docs/grafana/latest/panels-visualization...

It seems to miss some aggregation stuff, but also it's improving every time I check. I wonder if anyone's used it in anger yet and how far is it from replacing datadog or honeycomb.

tempo still feels very much: look at a trace that you found from elsewhere (like logs).

with so much information in traces and the pure volume, the aggregation really is the key to actionable info out of a tracing setup if it's going to be the primary entry point.

I've not. Honestly, I'm not in the market for tool shopping at the moment, I need another honeycomb-style moment of "this is incredible" to start looking again. I think it would take "Honeycomb, but we handle metric rollups and do logs" right now.
You can also check out SigNoz - https://github.com/SigNoz/signoz. It has logs, metrics, and traces under a single pane. If you're using otel libraries and otel collector you can do a lot of correlation between your logs and traces. I am a maintainer, and we have seen a lot of our users using signoz to have the ease of having three signals in a single pane.