[jordigh]

One thing that pains me about this kind of zoo of problems is that people often have the takeaway, "floating point is full of unknowable, random errors, never use floating point, you will never understand it."

Floating point is amazingly useful! There's a reason why it's implemented in hardware in all modern computers and why every programming language has a built-in type for floats. You should use it! And you should understand that most of its limitations are an inherent mathematical and fundamental limitation, it is logically impossible to do better on most of its limitations:

1. Numerical error is a fact of life, you can only delay it or move it to another part of your computation, but you cannot get rid of it.

2. You cannot avoid working with very small or very large things because your users are going to try, and floating point or not, you'd better have a plan ready.

3. You might not like that floats are in binary, which makes decimal arithmetic look weird. But doing decimal arithmetic does not get rid of numerical error, see point 1 (and binary arithmetic thinks your decimal arithmetic looks weird too).

But sure, don't use floats for ID numbers, that's always a problem. In fact, don't use bigints either, nor any other arithmetic type for something you won't be doing arithmetic on.

[gumby]

> Floating point is amazingly useful! There's a reason why it's implemented in hardware in all modern computers and why every programming language has a built-in type for floats.

I completely agree with you even though I go out of my way to avoid FP, and even though, due to what I usually work on, I can often get away with avoiding FP (often fixed point works -- for me).

IEEE-754 is a marvelous standard. It's a short, easy to understand standard attached to an absolutely mind boggling number of special cases or explanation as to why certain decisions in the simple standard were actually incredibly important (and often really smart and non-obvious). It's the product of some very smart people who had, through their careers, made FP implementations and discovered why various decisions turned out to have been bad ones.

I'm glad it's in hardware, and not just because FP used to be quite slow and different on every machine. I'm glad it's in hardware because chip designers (unlike most software developers) are anal about getting things right, and implementing FP properly is hard -- harder than using it!

[ogogmad]

> And you should understand that most of its limitations are an inherent mathematical and fundamental limitation, it is logically impossible to do better on most of its limitations

You can do exact real arithmetic. But this is only done by people who prove theorems with computers - or by the Android calculator! https://en.wikipedia.org/wiki/Computable_analysis

Other alternatives (also niche) are exact rational arithmetic, computer algebra, arbitrary precision arithmetic.

Fixed point sometimes gets used instead of floats because some operations lose no precision over them, but most operations still do.

[jordigh]

In my opinion, that's in the realm of "you can only delay it". Sure, you can treat real numbers via purely logical deductions like a human mathematician would, but at some point someone's going to ask, "so, where is the number on this plot?" and that's when it's time to pay the fiddler.

Same for arbitrary-precision calculations like big rationals. That just gives you as much precision as your computer can fit in memory. You will still run out of precision, just later rather then sooner.

[ogogmad]

> Same for arbitrary-precision calculations like big rationals. That just gives you as much precision as your computer can fit in memory. You will still run out of precision, later rather then sooner.

Oh, absolutely. This actually shows that floats are (in some sense) more rigorous than more idealised mathematical approaches, because they explicitly deal with finite memory.

Oh, I remembered! There's also interval arithmetic, and variants of it like affine arithmetic. At least you know when you're losing precision. Why don't these get used more? These seem more ideal, somehow.

[genneth]

Because the interval, on average, grows exponentially with the number of basic operations. So it quickly becomes practically useless.

[gugagore]

If x is the interval [-1, 1], the typical implementation of IA will

evaluate x-x to [-2, 2] (instead of [0, 0], and

evaluate x*x [-1, 1] instead of [0, 1].

Therefore the intervals become too conservative to be useful.

[lanstin]

I wouldn't call computable reals the reals. They are a subset of measure zero. Perhaps all we sentient beings can aspire to use, but still short of the glory of the completed infinities that even one arbitrary real represents.

One half : )

[saagarjha]

These are only relevant in some circumstances. For example, a calculator is typically bounded in the number of operations you can perform to a small number (humans don’t add millions of numbers). This allows for certain representations that don’t make sense elsewhere.

[zokier]

> 3. You might not like that floats are in binary, which makes decimal arithmetic look weird. But doing decimal arithmetic does not get rid of numerical error, see point 1 (and binary arithmetic thinks your decimal arithmetic looks weird too).

One thing that I suspect trips people a lot is decimal string/literal <-> (binary) float conversions instead of the floating point math itself. This includes the classic 0.1+0.2 thing, and many of the problems in the article.

I think these days using floating point hex strings/literals more would help a lot. There are also decimal floating point numbers that people largely ignore despite being standard for over 15 years

[jordigh]

The only implementation of IEEE754 decimals I've ever seen is in Python's Decimal package. Is there an easily-available implementation anywhere else?

[zokier]

I don't think Pythons Decimal is ieee754, instead its some sort of arbitrary precision thingy.

GCC has builtin support for decimal floats: https://gcc.gnu.org/onlinedocs/gcc/Decimal-Float.html

There are also library implementations floating around, some of them are mentioned in this thread: https://discourse.llvm.org/t/rfc-decimal-floating-point-supp...

decnumber has also rust wrappers if you are so inclined

[jordigh]

Python's decimal absolutely is IEEE 754 (well, based on the older standard, which has now been absorbed into IEEE 754):

https://github.com/python/cpython/blob/main/Lib/_pydecimal.p...

Cool, didn't know that gcc had built-in support. But is it really as incomplete as it says there?

[zokier]

Huh, I didn't know it was that close, I'll grant that. But I'd say still no cigar.

One of the most elementary requirements of IEEE754 is:

> A programming environment conforms to this standard, in a particular radix, by implementing one or more of the basic formats of that radix as both a supported arithmetic format and a supported interchange format.

(Section 3.1.2)

While you could argue that you may configure Decimals context parameters to match those of some IEEE754 format and thus claim conformance as arithmetic format, Python has absolutely no support for the specified interchange formats.

To be honest, seeing this I'm bit befuddled on why closer conformance with IEEE754 is not sought. Quick search found e.g. this issue report on adding IEEE754 parametrized context, which is a trivial patch, and it has been just sitting there for 10 years: https://github.com/python/cpython/issues/53032

Adding code to import/export BID/DPD formats, while maybe not as trivial, seems still comparatively small task and would improve interoperability significantly imho.

[zokier]

> One thing that pains me about this kind of zoo of problems is that people often have the takeaway, "floating point is full of unknowable, random errors, never use floating point, you will never understand it."

> Floating point is amazingly useful!

Another thing about floats is they are for most parts actually very predictable. In particular all basic operations should produce bit-exact results to last ulp. Also because they are language independent standard, you generally can get same behavior in different languages and platforms. This makes learning floats properly worthwhile because the knowledge is so widely applicable

[jsmith45]

>In particular all basic operations should produce bit-exact results to last ulp.

As long as you are not using a compiler that utilizes x87's extended precision flaots for intermediate calculations, and silently rounding whenever it transfers to memory (That used to be a common issue), and as long as you are not doing dumb stuff with compiler math flags.

Also if you have any code anwhere in your program that relies on correct subnormal handling, then you need to be absolutely sure no code is compiled with `-ffast-math`, including in any dynamically loaded code in your entire program, or your math will break: https://simonbyrne.github.io/notes/fastmath/#flushing_subnor...

And of course if you are doing anything complicated with floating point number, there are entire fields of study about creating numerically stable algorithms, and determining the precision of algorithms with floating point numbers.

[carapace]

Floating point is a goofy hacky kludge.

> There's a reason why it's implemented in hardware in all modern computers

Yah, legacy.

The reason we used it originally is that computers were small and slow. Now that they're big and fast we could do without it, except that there is already so much hardware and software out there that it will never happen.

[dahfizz]

What replacement would you propose? They all have different tradeoffs.

[carapace]

(I just tried to delete my comment and couldn't because of your reply. Such is life.)

ogogmad made a much more constructive comment than mine: https://news.ycombinator.com/item?id=34370745

It really depends on your use case.

[astrange]

Turning all your fixed-size numeric types into variable-sized numeric types introduces some really exciting performance and security issues. (At least if you consider DoS security.)

I think fixed-point math is underrated though.