Hacker News new | ask | show | jobs
by ChuckMcM 17 days ago
I worked in Systems Validation at Intel when the 8087 was current. Intel had an engineer dedicated to validating customer bug reports and reproducing them. Day in, day out, that's pretty much all he did. Sooooo many corner cases, and so many opinions on what the 'right' thing to do was when you lost precision[1].

[1] I'd say that over half of the bug reports were people who were annoyed that doing fp instructions in one order got them the right answer but in another order got them the wrong answer.

2 comments

Mathematicians vs Computer / Systems Engineers. The machine only has so much space, so it's best to imagine every value also has a corresponding error range attached and that managing the growth of that error range so that it remains under the target value is key.
And that operations combine those error bars in unintuitive (until you develop that intuition… but then does that still count as intuition?) ways.
IMO, the way the error bars combine is very intuitive. You are really just rounding to 6 or 12 sig-figs after every operation.

People just seem to get really hung up on the point that error bars exist in the first place, and combine.

I suspect it has a lot to do with the way that rounding is taught in school. It's absolutely hammered into use that you should never round until the very end, otherwise you lose precision.

We had a whole course at university (early 1990s) about stability in numerical calculations and simulations. Which was basically about rounding errors, plus some higher level mathematical transformations you could apply to stop errors from accumulating. I remember essentially nothing about it except that the lecturer insisted everyone use FORTRAN 77 (including uppercase and punch-card formatting) for the exercises.