Hacker News new | ask | show | jobs
by lights0123 75 days ago
Nice! There's also zstd's flush ability that I've used for streaming robotics data. You can write data and flush it over the network for realtime updates, but the compression stream stays open so it can still reference past messages. This means messages get smaller over time so you don't need to share a dictionary ahead of time. I'm not aware of other compression algorithms that have flushing capability like this.

> binary data to connected clients in tiny messages, each saying “field 5 on object X is now 123”

I wonder how Meta's newer, format-understanding OpenZL would do. I imagine its schemas could be auto-generated from protobuf.

1 comments

Ah, I have not looked into that, zstd keeps giving.