[danpalmer]

Honeycomb has one trick that it does very well, the high dimensionality analysis. Other than that Honeycomb doesn't do a lot, their alerting is basic, their dashboards are basic, etc.

Datadog doesn't do that specific feature as well (it has alternatives), but it also has so many other features that all tie together very nicely: metrics, logging, events, very good dashboards, analysis notebooks, alerting, SLOs, performance monitoring, trace analysis, security monitoring. It's a really extensive product.

When it comes to general observability, I'm a strong believer that you need a wide range of different views – just logs aren't enough, just metrics aren't enough, etc. I've worked in a team trying to use Prometheus for everything and there was so much friction, whereas with Datadog there has always been a way to achieve something.

I think Honeycomb is a good feature that should be bought by a company like Datadog and integrated into a wider more mature feature set.

[backslash_16]

Can you help me out and define what dimensionaltity means for data? And what high dimensionaltity is?

I’m somewhat familiar with monitoring/telemetry but haven’t heard that term before.

[danpalmer]

Sorry for the late reply. Dimensionality is just the number of dimensions a given record has.

For example, if you have a log line to represent an HTTP request completing it might have a server hostname that processed it, a path, and a request duration – 3 dimensions. High dimensionality is just having lots of dimensions.

Most monitoring systems aren't great at correlating between lots of dimensions, or cost a lot if you want to high cardinality, which high dimensionality contributes to. The cardinality is how many possible options there are for all of the dimensions together (so if you have 3 paths and 2 servers, those two fields have a cardinality of 6, or 6 different places you need to store your request duration for).

When you start getting lots of dimensions and lots of values for each dimension, things start getting expensive and you have to be quite restrained about what you choose to monitor. Maybe you decide not to track duration per server because you hope your servers are roughly the same, whereas you know that URL paths perform differently much of the time.

Honeycomb's great advantage is that they support this high dimensionality/cardinality really well. Not only do they not cost a lot to do it, but they also have a really nice UI for exploring this data and how different bits correlate together, without you having to know up-front what you want to look for. Honeycomb is expensive, but not prohibitively so for some engineering teams.

I'd still recommend Datadog _first_ because it does a lot more for a little less money, but if you're pushing the boundaries of what Datadog is capable of with tracing through distributed systems then Honeycomb enables the next step.