[isogon]

Repurposing flags not always well-motivated, but one legitimate reason to do this is the memory (and particularly cache) footprint.

Often flags are local to a particular object. If there are lots of such objects, you want each to take as little space as possible. You should check out the contortions linux devs go through to make struct page small [0]. This is important, because there is one such struct per page of physical memory. The memory use is a near-constant percentage of your total memory, and you wouldn't want it to be any larger than necessary.

Even when there are not a lot of these objects, in low-latency software it's important to hit the cache. Your program should always just be as compact in memory as possible.

Semantically flags are booleans (is proposition P true of this object). They are stored compactly as bitsets, often implicitly, say:

    #define FLAG_1 0x01
    #define FLAG_2 0x02
    /* ... */
    #define FLAG_8 0x80

   struct order {
       u32 qty;
       u16 id;
       u8  type;
       u8  flags;
   };

This struct will fit into 8 bytes. This is great, as you probably won't waste space to alignment in many cases -- 8 is a good multiple. But if you wanted to add FLAG_9 here, your flags would become a u16, and your struct would, frustratingly, stop fitting into 8 bytes. To avoid this, one might repurpose flags.

Another example of this is intrustive flagging, using, for example, the high or low bits of a pointer aligned to 2^n bytes. If you run out of bits there, not much you can do.

[0] https://github.com/torvalds/linux/blob/master/include/linux/...

[pclmulqdq]

This is pretty much why flags get repurposed. It's also important to mention that things like JSON and protobufs are too expensive for HFT, so you are likely going to be sending structs over the wire. Repurposing flags lets you change a wire format with a lot less friction than adding a byte to a struct. Essentially, it lets you change the minor version number on a protocol and only recompile the endpoints without changing the major version number and recompiling everything.

[aaronharnly]

Thank you both!

[lordnacho]

This is what I thought as well, and I'm in the field. It's legit to try to fit everything in cache.

However, you can still do something about making this safe. For instance the program could do some sort of version check on startup and panic if things weren't correct.

There's a bunch of stuff that needs to be done before the program reaches its low latency steady state, where speed doesn't matter. Might as well add checks there to make sure things are correct.