Hacker News new | ask | show | jobs
by thedance 2269 days ago
You can beat the pants off this, even though it beats the standard proto library. Most of the time here is being spent calling out-of-line functions through Go's stack calling convention because the Go compiler hates inlining. For example, DecodeTagAndWireType is CALLing DecodeVarint, even though that's really where you want it to have been inlined.

If your use case is really simple, and you want the fastest possible proto decoding, you can hand-roll it pretty easy.

  message point { fixed32 x = 1; fixed32 y = 2; }
Then your code is structured such as

  var x uint32
  var y uint32
  for len(buf) > 0 {
    switch buf[0] {
    case (1<<3) & 5: // Field 1, type fixed32
      x=binary.LittleEndian.Uint32(buf[1:])
      buf = buf[5:]
    case (2<<3) & 5: // Field 2, type fixed32
      y=binary.LittleEndian.Uint32(buf[1:])
      buf = buf[5:]
    default:
      // Freak out somehow
  }
Improve the exception hardness of this code to the extent that it suits your use case.
3 comments

Yeah thats a good point. The use-case I wrote this for involved decoding a protobuf message that has only 3 fields, but where the total size is several MiB. For my workload this optimization is meaningless, the most important thing is to avoid allocating huge slices, but for lots of small protobufs like the one you defined a hand-rolled solution will definitely be faster.

Do you think I could just manually inline the decoding code as much as possible in the molecule library, or does it need a different API?

Also if you open a P.R I'll merge it :)

Heh OK, see my other post for the simplest possible improvement.
Example: hoisting the condition `if buf[i] < 0x80 ...` out of DecodeVarint into DecodeTagAndWireType speeds up the included microbenchmark by over 10% on my machine, just because it eliminates the function call for tags 1-15 (these are the magical tags, if you are using protobufs in a performance-sensitive context).
Don’t you need a bounds check?

    for len(buf) > 4 {
Sure, that's why I put that last remark. But you might already know before hitting this block that the message is perfectly formed.