[capyba]

Forgive me - much of the article went over my head so I’m trying to understand: this article is excellently written, but I’m struggling to understand why a `Vec<u8>` wouldn’t have sufficed? And disregarding the first question, is there a way to extend this to floats, for instance if I have a collection of values that I know will never exceed the bounds of +/- 10.0 with 1e-6 precision, can I use this technique for more efficient storage of them?

[dathinab]

> wouldn’t have sufficed?

most times it will suffice and be the recommended solution (less complexity; no ups I though I only need 3bit but at runtime needed 4bits situations; one less dependencies to review for soundness issues / bugs with every update)

while using Vec<u64> with a 3bit number is a pretty strange/misleading example (as you can use a Vec<u8>) my guess is that they used it because many packed vectors operate on u64s internally and allows up-to 64bit values.

anyway while often it doesn't matter sometimes storing a u8 where you only need 3bit numbers or similar makes a huge difference (e.g. if with packaging you application state fits in memory and without it doesn't and start swapping)

you can do similar stuff for non integer values, but things get more complicated as you can't just truncate/mask floats in the same way you can do so for integers. Further complications come from decimal based precision bounds and the question of if you need perfect precision in the whole number range (floats aren't evenly spaced).

A common solution is to use fixed point floats or in general convert your floats to integers. For example, assuming a perfect precision requirement for you range, you could store a number in range [-1e7; 1e7] and by context/in code remember there is a implicit missing div 1e6. Then you can store it in a 25 bit integer and bit package it (log2(1e7).ceil()+1 sign bit).

[lukefleed]

The other replies nailed it, but I'll add my two cents.

Vec<u8> may be the right call most of the time for most use cases. This library, however, is for when even 8 bits compared to 4 is too much. Another example, if all your values fit in 9 bits, you'd be forced to use Vec<u16> and waste 7 bits for every single number. With this structure, you just use 9 bits. That's almost a 2x space saving. At scale, that makes a difference.

For floats, you'd use fixed-point math, just like the other replies said. Your example of +/- 10.0 with 1e-6 precision would mean multiplying by 1,000,000 and storing the result as a 25-bit signed integer. When you read it back, you just divide. It's a decent saving over a 32-bit float.

[Dylan16807]

If you need absolute precision then floating points aren't going to be optimal. In particular your constraints need pretty much all the precision a 32-bit float has to offer, and you still need a handful of exponent bits, so your total savings would be like 4 bits.