[patrec]

When does this need arise? Well, otherwise inverting a value can change it's sign and in particular inverting -∞ twice will give you +∞, and being off by "2∞" is a pretty large error for a lot of computations ;)

You can end up with zeros and infinities pretty easily because you overflow or underflow the range of floating point precision, and generally you want something sensible to happen in typical cases, even if some common arithmetic identities necessarily break down.

I would actually like a true signed zero (or rather "epsilon" value ), so -0 and +0 as distinct from "normal" 0 which is truly unsigned, neither positive nor negative. The former two would only arise from underflow, and the reason this is useful that if you underflow from below zero and invert that you want to get -∞ and if you underflow from above zero and invert that you want to get +∞. Inverting a signless zero should give NaN (instead it gives +∞, which is nonsense in basically any case where the domain is not inherently the non-negative reals already and the 0 did not come about by and underflow; in particular 1/0 should be NaN).

If anyone knows why this design was not chosen and what fundamental downsides it has, I'd love to hear it. Obviously representing three zeros is a tad more annoying, but IEEE754 has a lot of stuff that's annoying implementation wise but was added for nicer numerical behavior (e.g. denormals, and of course various "global" rounding modes etc. which probably qualify as a mistake in retrospect).

[Const-me]

> If anyone knows why this design was not chosen and what fundamental downsides it has, I'd love to hear it.

No fundamental ones, but a few practical.

32-bit numbers have 2^32 unique values, the number is even. Your approach makes the range asymmetrical like it happens with integers.

The range for 8-bit signed integers is [ -128 .. +127 ]. On ARM NEON there’re two versions of integer negate and absolute instructions, some (like vqnegq_s8 or vqabsq_s8) do saturation i.e. transform -128 into +127, others (vnegq_s8, vabsq_s8) don’t change -128. Neither of them is particularly good: the saturated version violates -(-x) == x, non-saturated version violates abs(x)>=0. Same applies to the rest of the signed integers (16, 32, 64 bits), an all modern platforms.

With IEEE floats the range is symmetrical and none of that is needed. Moreover, PCs don’t have absolute or negate instructions, but they instead have bitwise instructions processing floats, like andps, orps, xorps, andnps, they allow to flip, clear or set just the sign bit, very fast.

Another useful property of IEEE representation is that for two intervals [ 0 .. FLT_MAX ] and [-FLT_MAX .. -0.0f ] sort order of floats corresponds to [inverted] sort order of 32-bit integers.

[patrec]

> Your approach makes the range asymmetrical like it happens with integers.

Not necessarily since you've got (a lot of different) NaNs anyway. For the sake of argument, you could give up one one of them and make it positive zero (since this representation would be unnatural, it would slow stuff down, just as non-finite values do on many CPUs. Wouldn't matter that much since signed zeros would only arise from underflow).

> Another useful property of IEEE representation is that for two intervals [ 0 .. FLT_MAX ] and [-FLT_MAX .. -0.0f ] sort order of floats corresponds to [inverted] sort order of 32-bit integers.

I'm aware, but as you correctly note this only works in the right direction for unsigned values. And it's just not that important a benefit, I'd much rather have my calculations come out right than being able to sort positive floating point numbers with an integer sort routine.

[Const-me]

> you could give up one one of them and make it positive zero

What do you expect to happen when you set the sign bit of that number? A possible answer to that is "negative zero", and now you have 2 separate encodings for negative zeroes: one of them with exponent 0, another one 0xFF, and they behave slightly differently.

[patrec]

If you manually screwed around with the most significant bit of the underlying bit pattern (rather than just writing -x like any normal person) I'd expect to get a NaN -- assuming of we keep an explicit sign bit in the representation at all. It would obviously make hardware implementation of negation more complex (and thus slower) but I don't think there is any conceptual problem.

[Const-me]

PC hardware doesn’t have hardware implementation of negation.

Possible to do in 2 instruction, xorps to make zero, then subps to subtract. Combined, they gonna take 4-5 cycles of latency (xorps is 1 cycle, subps is 3 cycles on AMD, 4 cycles on Intel).

If you do that a lot, a single xorps with a magic number -0.0f gonna negate these floats 4-5 times faster. People don’t pay me because I’m a normal person, they do that because I write fast code for them :-)

On a serious note, I’d rather have the current +0.0 and -0.0 IEEE values to be equal and be the exact zero, and make another one with 0xFF exponent encoding inexact zeroes, +0.0f or -0.0f depending on the sign bit.

Or another option, redefine FLT_MIN to be 2.8E-45, and reuse the current FLT_MIN, which is 1.4E-45 / 0x00000001 bit pattern, as inexact zeroes.

[bluecalm]

I wonder if it would be useful to have a signed integer that has symmetric range and the one that is left is used as NaN. Overflows would set it to NaN for example. Then again, once that's on the table it's very tempting to steal two more values for +/- inf. I think it's very useful to have full range unsigned ints but signed ones could have range reduced to make them less error prone.

[phkahler]

Posits handle this by using the smallest non-zero number where floating point would go to zero. They also use the largest represent able number instead of infinity. These exist for both positive and negative numbers. At least that's the way I read it.

[dnautics]

Co-inventor of posits here, this is basically correct, there still is "infinity" in posits, it's strictly reachable by inverting 0 directly.

[patrec]

Having a single (unsigned) infinity (and a single zero!) seems cleaner in some ways (and I dimly seem to recall that some pre-ieee754 floating point hardware worked that way). On the other hand, having e.g. a neutral element for both max and min also seems pretty nice to have, although without infinities, the maximal and minimal floating point value will equally do the trick in most cases.

How do inequalities work for posit infinity? Is posit infinity both larger and smaller than any other posit?

[phkahler]

I thought that was essentially NaN and could also result from square root of negative numbers?

[afiori]

a cons of that is that there is no longer a distinction between positive and negative infinity

[jayd16]

So no then? As you say, epsilon should be used in this case.

[patrec]

I meant as in infinitesimal, not as in machine epsilon (which lacks the required properties) -- poor wording on my part. If you have numbers of impossibly large magnitude you probably also want corresponding numbers of impossibly small magnitude. You can do this in a nice, algebraically satisfying way with infinitely many such numbers (hyperreals), but I think if it weren't for the signless zero and positive zero conflation, IEEE754's way would be quite a reasonable finite precision approximation to this.

[amelius]

∞ is not a number. Using it as a number is a hack invented by mathematicians.

[steerablesafe]

∞ is not used as a number by mathematicians. Maybe by engineers.

[chithanh]

That is not entirely correct. Schmieden and Laugwitz for example developed in the 1950s a nonstandard Analysis which adjoins an infinitely large element (called Ω) to the natural numbers. The basic idea was a formula A(Ω) was true if A(n) was true for almost all finite natural n.

While it wasn't immensely useful going forward, it helped to clarify the use of infinity and infinitesimals in earlier work.

[steerablesafe]

I'm well aware of nonstandard analysis, but ∞ is still not a number there, even though there are infinitely many infinitely large elements .

[Spivak]

The extended complex plane is a space where inf is actually number and where division by zero is allowed.

Same with the extended real line.

Infinity is as much a number as it is useful to define it as such.

[steerablesafe]

Ah right, I forget that extending with a single infinity element is useful with complex numbers and with geometry. It's still not very common with the reals alone as +∞ and -∞ are reasonable to want as separate elements there, but it doesn't play nicely with 1/0 that way.

[prionassembly]

Ehh.

https://en.wikipedia.org/wiki/Extended_real_number_line

NB this is hardly nonstandard analysis.

[analog31]

That's right. It's a symbol. When you see it in an expression, you're probably expected to interpret it in light of a limit of some kind. It's just convenient to write it into an expression rather than use cumbersome limit notation all over the place.

Like many notational shortcuts, it's a hack supported by proof. ;-)

[taeric]

This explanation feels off, to me. Aren't all numbers symbols? Pi, e, 10, 0xA, 9.99999...?

[analog31]

In my view, numbers are numbers, and symbols are symbols. There's an agreement that a symbol represents a number, but there's not a one-to-one relationship between available symbols and available numbers. Normally this isn't a problem, and we treat them interchangeably. And indeed the distinction may only be a philosophical oddity or a matter for mathematicians. But I believe nonetheless that there is a distinction.

Now I was merely an undergrad math major, which means I topped out before learning this stuff in a formal way. But at my primitive level of understanding, I think of a number as something that behaves like a number within a given system. What I learned in my courses was how different kinds of numbers behaved: Whole numbers, reals, complex, vectors and tensors, etc. I remember reading a definition of "tensor" that was to the effect of: A tensor is something that behaves like a tensor, meaning that the important thing is the behavior.

Another post in this page expressed that we should be particularly cautious when dealing with numbers, symbols, and IEEE floats, notably to beware that IEEE floats and real numbers don't always behave the same. That was treated in one of my math courses, "Numerical Analysis." You could also get CS credit for that course, suggesting its practical importance.

[taeric]

I think the consequences of what you are saying makes sense. Would be neat to explore more of the idea. I was starting to find, recently, that it was better to think of numbers as symbols that follow operational rules. This view seems counter to that.

[analog31]

There is a view that math is "just" symbol manipulation. So I can't say your approach is wrong. Probably whatever works, works.

And you can usefully say things like "x is a number" without saying which particular number x is.

[bjourne]

Yes, in a way. What distinguishes irrational numbers from rational numbers is that all rational numbers can be represented by strings drawn from a regular language. For example, all strings generated by the regular language "-?\d+\.\d+?_\d+" (where "_" denotes the repeating decimal expansion as in 1/6 = 0.1_6) correspond to exactly one rational number and all rational numbers correspond to at least one string in this regular language. Irrational numbers (and other types of "numbers" such as +inf and -inf) cannot be represented by any such regular language.

[taeric]

I'm not entirely sure I follow. Aren't pi and e irrational numbers?

I also included .9_ as it is an easy trap to show we have two ways of writing 1 in standard decimal notation.

Please read this whole post as a question. I'm genuinely not clear on the distinction.

[bjourne]

Correct. Given the regular language I specified, each rational has an infinite number of matching strings: 1.0 = 1.00 = 1.000 = 0.9_99 = 1.0_0 etc. The point is that for every rational number you can think of, I can show you at least one string in my regular language to represent that number.

According to the finitists, this is a defining feature of a "number". Since the same can't be done for irrational numbers finitists conclude that irrational "numbers" aren't numbers. You probably agree that all numbers are (or can be represented by) symbols, but that not all symbols are numbers. So how do we distinguish symbols from numbers?

[gefh]

But not all symbols are numbers.

[taeric]

Why not? One of the profound effects of computers is that we have moved computing to be able to work with images. Sounds. Etc.

I get that not all have simple algebraic concepts.

I get the impression I'm corrupting parts of I am a Strange Loop? Would love to read more on this.

[colejohnson66]

I was taught that you’re supposed to read $y = \inf$ as:

    y = lim_{x->\inf} x

Basically, $y$ isn’t necessarily infinity, but just a number larger than you could ever write. The number at the end of the number line if it existed.