Hacker News new | ask | show | jobs
by dilatedmind 1996 days ago
did you use any kind of message framing? Got bit by this at a previous job where we needed to change the message format to improve compression. Wound up figuring something out, but would have been easier if we had reserved a byte for versioning.
1 comments

I'm not quite sure what you mean by message framing.

If you mean marking messages as having been compressed, then absolutely yet. The Pub/Sub messages were tagged with `compress=true|false` so we could update downstream and upstream independently.

If you mean buffering log messages into a batched 'frame', then yes we did do this. We were taking about 500 log entries and compressing them into a single message, which was part of why the compression was so effective.

If you mean something different, then I'm at a loss and very interested in what you meant by the term!