I think it has many good parts. There are waaaaay worse formats I have worked with over the years. Like COM, java RMI, many variants of SOAP, handcrafted json in countless variants…
not self describing. if it was just the field names I could deal with that, but even the values are ambiguous, since the same type is used for Bytes and embedded Messages. the worst part is the wire type integer has two unused values, so they easily could have added a wire type for embedded messages
Self-describing is point-less for serializations. There is a great deal of history here. ASN.1 has self-describing encoding rules such as BER/DER/CER, XER (XML), JER (JSON), and GSER (never mind), and it has non-self-describing serializations like PER (packed encoring rules) and OER (octet encoding rules). XML and JSON are self-describing, naturally. FastInfoSet is a PER-based fast encoding for XML, because it turns out that XML is slow (imagine that). XDR is a non-self-describing serialization format that resembles OER but with 4-octet alignment. Flat buffers is essentially an OER-ish encoding for the same IDL as protobufs, and is much better than protobufs.
It would be nice if the next serialization format either is truly original or just solves problems that somehow none of the many existing schemes do.
> Self-describing is point-less for serializations
you couldn't be more wrong. what happens when you lose the schema, or never had access to it in the first place? think from the point of view of reverse engineering
I imagine when someone is choosing a serialization solution they probably don't really care about people trying to reverse engineer it... And if they did, they would just make schemas available instead.
And if you lose your own schema, then you probably have more serious underlying problems.
I'm not sure that's a helpful way to look at things.
There are tradeoffs. Can you elaborate more about what aspects of a TLV encoding you find problematic? Is it decoding speed? The need to copy the encoded value into a native value in order to make use of it? Something else?
TLV encodings are always redundant and take more space than non-TLV encodings. Therefore they are a pessimization. As well, definite-length TLV encodings require two passes to encode the data (one to compute the length of the data to be encoded, and one to encode it), thus they are a) slower than non-TLV binary encodings, b) not on-line for encoding.
Yes all those are indeed pessimizations during encoding but are features when decoding: the decoder can skip decoding fields and tolerate unknown fields.
Now, you may disagree that tolerating unknown fields is a features (as many people do), but one must understand the context where protobuf has been designed, namely the situation where it takes time to roll out new versions of binaries that process the data (either in API calls or on stored files) and thus the ability to design a schema evolution with backward and forward compatibility is worth a few more cycles during encoding.
Not all users have that need and hence there exist other formats, but I wouldn't dismiss the protobuf encoding as flatly "wrong" just because you don't have the requirements it has been designed for.
such an awful format, I wish people would stop using it