[yalok]

There are research papers where even 1 bit (not floating point) was enough, with some quality loss.

4 bits is effectively 16 different float point numbers - 8 positive, 8 negative, no zero and no NaN/inf. 1 bit for sign and 3 bits for exponent, 0 bits for mantissa, mantissa is implied to be 4. It’s logarithmic - representing numbers in the range from -4^3 to 4^3, smallest numbers are 4^-3.

[phh]

Thanks. First source i see for what fp4 is. Gotta say I'm surprised: I would have chosen to lose one value, but have a zero. (though I have no doubt those people are much more clever and knowledgeable than I am)

[omikun]

If the weight is zero it doesn’t need to exist

[carlmr]

>1 bit (not floating point)

I like how you specified that it's not floating point.

[s_m_t]

Thanks, I was thinking that zero, negative zero, inf, negative inf, and the NaN's were included like in IEEE 754