[ynniv]

cheald has an excellent comment regarding MessagePack, JSON, and Protocol Buffers in the post from 15 hours ago: http://news.ycombinator.com/item?id=4091051

   MessagePack is not smaller than gzip'd JSON
   MessagePack is not faster than JSON when a browser is involved

Other comments from that post include:

   MessagePack is not human readable
   MessagePack has issues with UTF-8
   Small packets also have significant TCP/IP overhead

Really, anyone who hasn't read the other comments should: http://news.ycombinator.com/item?id=4090831

[frsyuki]

Although the original blog post focuses on JavaScript and browsers, MessagePack itself doesn't mainly focus on them.

A major use case of MessagePack is to store serialized objects in memcached. A blog post written by Pinterest describes this use case (http://engineering.pinterest.com/posts/2012/memcache-games/). They use MessagePack with Python which is faster than one with JavaScript. They could store more objects in a server without performance declination (e.g. gzip).

It's true that MessagePack is not always faster than JSON (e.g. within browsers), and it's not always smaller than other serialization methods (e.g. with gzip compression). So we should consider that which serialization methods should I use for "my" case.

There are also general tendency which is helpful to select MessagePack or JSON:

    MessagePack is faster to serialize binary data such as thumbnail images.
    MessagePack is better to reduce overheads to exchange small objects between servers.
    JSON is better to use it with browsers.

[alexgartrell]

> They could store more objects in a server without performance declination (e.g. gzip).

The performance declination argument is bullshit. Network's a million [0] times slower than gzip.

Truth be told, once you're on the network, you're already screwed w.r.t. most serialization. The only thing efficient compression/decompression is going to buy you is lower CPU (memcached servers run at like 2% CPU util, even under heavy load [1]).

Memcache at Facebook actually uses the ascii protocol, and the memcached implementation is a braindead strtok parser (some of our other stuff uses ragel -- you'll have a hell of a time out optimizing ragel with the right compiler flags -- I've tried and failed).

Just use whichever serialization format has the best API, because I can say with near certainty that it's not going to be a perf problem for you if you're touching disk, network, etc.

[0] Obviously a made up number, but it's way slower. Especially if you're unlucky and lose a packet or something.

[1] With the exception of weird kernel spin lock contention issues, which can happen if you're not sharding your UDP packets well and trying to reply from 8+ cores on 1 UDP socket. You probably aren't.

[catwell]

I +1 that. I have working experience with MessagePack and I can confirm it works for the following use cases:

* RPC communication between servers where binary data is exchanged and its structure is not always the same (ie. difficult to use something that requires an IDL).

* Serialization and storage of objects that will be sent over the network (note: you can batch MessagePack objects just by concatenating them).

* Communication between a server and a native mobile application. Native applications live in a binary world whereas Web applications live in a text-based world where JSON is better.

The human readability argument is poor: the JSON that is sent over a network is not usually human readable, so you would use a prettyfier to read it anyway. Moreover a MessagePack message is standalone / self-describing, ie. you don't need an IDL description to read it. So in both case, reading the message is just adding another block to a pipeline...

[jamesaguilar]

That test you linked to which claims that messagepack is 4x faster seems to rely on the serialized text staying in-process. The vrefbuffer is only zero-copy as long as you don't need to send it to any API which reads strings or char buffers (e.g. any RPC or network-oriented mechanism). Am I reading it right?

[latch]

I've used it to store blobs of data in databases, specifically when space tends to directly equate to memory (like redis and mongodb). You can take this pretty far and apply different serialization or compression algorithms based on the data (and store a field that says which approach was used for when you deserialize it).

Using this from the browser is not the first thing that came to my mind.

[SeoxyS]

Storing a message-pack serialized object in MongoDB is silly, because Mongo is essentially storying BSON-serialized object on disk. You're double-serializing data in two competing formats.

[latch]

BSON isn't meant to be compact, it's meant to be quick and efficient to serialize and deserialize. You store it inside MongoDB as bindata, and you'll save space.

[frsyuki]

MessagePack includes a concept named "type conversion" to support types which are not supported by its wire format. With the concept, we can serialize/deserialize user-defined classes as well as strings with encodings.

So far, MessagePack for Java, C++ and D implement the concept.