| So for reference to everyone, the way a fixed-width posit works is by unary-encoding the high bits of the exponent, then encoding the low bits if space is available, then encoding mantissa into whatever bits are left. Near 1.0 they look like a normal float. Maybe they have an extra bit or two of precision. Then as you get closer to 0 or infinity, every time the exponent field would run out you instead reset it, at the cost of one bit of precision. The main benefit is that they have a lot more dynamic range than an equivalent float. The downside is the further you go into that range, the less accurate your numbers are. They can also trade off accuracy near 1 for accuracy far from 1. Different exponent widths represent different points on this scale. At smaller bit sizes, they have a good mix of preserving precision while being quite hard to overflow. Overall they're not very different from standard floating point. Most of the bold claims from Gustafson are outside the scope of the posit itself. - Naively I would expect them to be kind of useful, if we ignore the issue of hardware support. Do neural networks need to represent extreme values with just as much precision as non-extreme values? And is the risk of overflow mitigated sufficiently at the same time? If so, yeah, the whole idea is useless. |