[piterrro]

Unless I dont understand that fully (which could be the case).

This idea could fly if downstream readers will be able to read it. Json is great because anything can read it, process, transform and serialize without having to know the intrisics of the protocol.

Whats the point of using binary, columnar format for data in transit?

[arccy]

better compression https://opentelemetry.io/blog/2023/otel-arrow/

You don't do high performance without knowing the data schema.

[odie5533]

Is Arrow better than Parquet or Protobuf?

[theLiminator]

Arrow is an in-memory columnar format, kinda orthogonal to parquet (which is an at-rest format). Protobuf is a better comparison, but it's more message oriented and not suited for analytics.

[datadrivenangel]

Not having to write to disk is great, and zero-copy in memory access is instant...

[arccy]

the blog post comparison is against OTLP which is protobuf