Hacker News new | ask | show | jobs
by IshKebab 1419 days ago
You're referring to the fact that Capn Proto claims to be zero copy. That doesn't mean there is no serialisation overhead.

In my experience with Capn Proto, the vast majority of the time the zero copy feature is pointless. The Capn Proto C++ APIs are extremely unergonomic so 99% of the time you end up copying the data into your internal nice C++ structures anyway, completely giving up zero copy.

I've used Capnp quite a lot and I really wouldn't recommend it. It's quite old and complex and the unpleasantness of the API alone is enough to put me off. I would pick Protobufs every day for small amounts of data.

For large amounts you are better off with SQLite or DuckDB.

2 comments

Capn'Proto isn't that bad. I've used the C++ quite a bit, and while the authors code style isn't to my taste it's fairly well designed (nice layering that lets you implement different use cases, like integrating the RPC framework with your own event loop).

There's also not much out there comparable to its capability passing RPC either.

Agree on the zero copy though, and the RPC framework does malloc a lot.

sqlite is not a serialization format though.
Yes it is. They even have a page about it: https://sqlite.org/appfileformat.html

It's a format. You can serialise data to it. What about it isn't a serialisation format?

Obviously I wouldn't recommend it for small amounts of data, e.g. for RPC calls. But when you need to store lots of data it's much better than JSON or binary JSON style formats.

But it makes a perfectly reasonable application file format. Coupled with schema evolution and an ORM it's pretty sublime.
It's a data format which can be expressed as a contiguous series of bytes and sent/stored; it's a serialization format.

This is not just nit-picking, it's been used precisely as such:

https://phiresky.github.io/blog/2021/hosting-sqlite-database...

Sorry, should have been specific. It's not used as a _wire_ serialization format.