Hacker News new | ask | show | jobs
by cozzyd 1968 days ago
That should be clear from looking at the assembly, shouldn't it? Don't think the floating point environment can change that but I could be wrong.

Floating-point rounding modes seem more likely to me. The author should be able to dump the floating point configuration to confirm, I'm sure.
1 comments
cperciva 1968 days ago
The same x86 instructions are used for arithmetic on float/double/extended types; the difference is a precision setting in the x87 control word.
link
cozzyd 1968 days ago
Well in most cases I've looked at generated assembly (not that often), the xmm registers are used even for scalar operations, which I thought was the default option for gcc on x86-64, but I suppose it might differ on different systems (or perhaps 32-bit mode was used for some reason).
link
cperciva 1968 days ago
Right, if you're using xmm regs you're getting double or single precision. Sometimes the x87 regs get used and that's when "accidentally computed with extra precision and then double-rounded" comes up.
link