[thamer]

A few years ago I worked on a service that had to stream data out using protobuf messages, in a single request that could potentially transfer several gigabytes of data. At the HTTP level it was chunked, but above that I used a protobuf message that contained data plus a checksum of that data, with the last message of the stream containing no data but a checksum of the entire dataset (a flag was included to differentiate between the message types).

This simple design led us to find several bugs in clients of this API (e.g. messages dropped or processed twice), and gave us a way to avoid some of the issues mentioned in this article. Even if you don't use HTTP trailers, you can still use them one layer above and benefit from similar guarantees.

[vidarh]

Inserting metadata in the protobuf itself seems like the obvious, simple solution to avoid having to depend on what the transport layer supports. Just defining a message to provide the metadata they wanted to insert in trailers would have avoided a whole lot of pain.