[andrew_rfc]

Cap’n Proto and Flatbuffers are not alternatives to Protocol Buffers. They solve entirely different issues and have distinct limitations.

They aren’t relevant to what Bebop is doing and thus no benchmark or comparison can be made between them.

[kentonv]

Sure they are. Cap'n Proto addresses a superset of Protobuf's use cases. Anywhere where you might use Protobuf, Cap'n Proto will work fine. In certain use cases (like mmaping large files or exporting complex APIs over RPC) Cap'n Proto is much better than Protobuf, but it's not bad at the other use cases.

The main limitation of Cap'n Proto compared to Protobuf is the ecosystem -- missing or poor-quality implementations in many languages, limited tooling, etc. Admittedly this is probably a showstopper for most users. It's also the hardest thing for any new contender to solve.

With all that said I would tend to agree that benchmarks are probably pointless. I've spent a lot of time benchmarking serialization and one thing I know is that benchmark results will vary wildly depending on the use case. A benchmark of an example/toy use case isn't really indicative of performance in a real use case.

(I'm the author of Cap'n Proto. I don't know much about Flatbuffers so can't comment there.)

[andrew_rfc]

Apologies if my initial comment came off the wrong way; when I say “not alternatives” I was injecting bias into that statement since I normally use Cap’n Proto on embedded and memory constrained systems (amazing library btw.)

When I created Bebop years ago I tried to benchmark it against Cap’n Proto, but the lack of a semi-decent web or C# implementation mentally made me recategorize.

So I suppose I’m just comparing what’s closest in terms of ecosystem support rather than a purely functional level.

[kentonv]

Yeah that's fair, it does seem like Bebop and capnp are targeted at different use cases, even if they're both Protobuf alternatives.

It obviously doesn't make sense to benchmark a JavaScript Bebop implementation against a C++ Cap'n Proto implementation, nor does it make sense to compare against a JS Cap'n Proto implementation written by a third-party contributor that is incomplete and unmaintained... so there isn't really a sensible comparison to make.

[bfrog]

I'm not sure how you can say they aren't alternatives, they directly market themselves as alternatives. The first graphic on the capnproto site compares it to protobufs and was itself created by one of the original protobuf authors as I understand it (Kenton).

Flatbuffers immediately compares itself to protobufs as well on the front page.