[aero142]

The world of observability seems to be converging on some key ideas. I think Vector's data model where they convert logs into structured events is a key idea that wasn't always obvious. I think the remaining big idea is what do do about high cardinality data. Most solutions pre-aggregate the data which does't tolerate high cardinality tags. Solutions like Honeycomb and Datadog logs are stream based and tolerate high cardinality tags, but with limitations on what can be done with it. It will be interesting if the streaming based solutions become the final standard. Vector warns about high cardinality labels, just like others do. I'm not sure if that is a limitation of Vector or just the Sink.

I think the interesting part is the overlap between observability metrics for operations and expanding in to BI metrics with the same tools.

[chasers]

> I think the interesting part is the overlap between observability metrics for operations and expanding in to BI metrics with the same tools.

I built Logflare so you could get your structured logs into BigQuery directly, so now you this :)

Hopefully we'll have a Vector sink soon, but until then I think they support POSTing batches to HTTP endpoints so you can do that and we'll take any JSON and go straight to BQ with it after migrating your schema for you (automatically based on the incoming payload shape).