Hacker News new | ask | show | jobs
by chrisseaton 2381 days ago
> due to using floats

No they weren't using floats. The article you've linked to says this really clearly.

> This calculation was performed using a 24 bit fixed point register

1 comments

http://www.cs.unc.edu/~smp/COMP205/LECTURES/ERROR/lec23/node...

They meant fixed precision not fixed point math.