Hacker News new | ask | show | jobs
by tijsvd 2004 days ago
But then all that must come with bookkeeping, which brings its own cost.

Take a look at an implementation like Prost, for Rust. It's very similar to what I did (10 years ago by now). Everything is just inline, except when messages can be recursive (which should be rare for most protocols).
2 comments
haberman 2003 days ago
> Everything is just inline, except when messages can be recursive (which should be rare for most protocols).

Many messages are have lots of optional sub-message fields, and set only a few of them in any given message. These messages would be huge if everything is inline (especially if the same thing happens in those sub-messages).

I agree that inlining all sub-messages works great for dense schemas, but it assumes too much about the schema to be a good design for a general-purpose proto library I think. Also maps and repeated fields can never be inline.

link
tijsvd 2003 days ago
Yes, this comes back to use case.

So it's great that there are different implementations for different use cases. It helps that the wire format is simple and well-documented.

link
kentonv 2003 days ago
The bookkeeping is not that hard... the pointer is null until it is first allocated, then it remains non-null, while a separate boolean indicates whether the sub-message is actually present in the parent.

The big problem is bloat in memory usage if you parse many differently-shaped messages, requiring the app to implement hacks like only reusing a particular object a certain number of times.

link