Hacker News new | ask | show | jobs
by soylentgraham 1322 days ago
Have you stepped through protobuf processing code? There's a lot of special cases, ifs, branches here and there. Protobufs within protobufs. Its not like its a size, then 100 floats packed together, theres more overhead than youd think. (Not to mention the client side allocations etc etc) I use protoc compiled to wasm for protobufs and it is fast, but theres a lot of wasm overhead to execute that code.

Json parsing is also a lot of special cases, error testing, but the v8 team has spent a huge amount of time optimising json parsing (theres a few blog posts on it). Im not assuming either way, but it's definitely as cut and dry as one would assume.

3 comments

Stepped through? Yes..as I hinted, coming from an embedded environment, and measured compared highly optimized json parsing code (that even had much limitations, like very limited nesting, no lists) vs nanopb => clear winner on all points (memory reqs, performance, encoded size) - which is really not that surprising?
There are two ways to encode a repeated field (100 floats, but could also be any size up to the limits of repeteating fields): "Ordinary (not packed) repeated fields emit one record for every element of the field." That means type, value, type, value, etc"

However, "packed" fields are exactly a length followed by a byte array of the typed data. This was an oversight in original proto2 which is unlikely to be corrected, but packed the default in proto3.

100 (or any N) floats prefixed by a size is exactly what you would get from `repeated float f = 1 [packed=true];`