[StavrosK]

How do you test it by brute force? How is the algorithm supposed to know what the right thing to return is, unless you rewrite the whole thing, correctly this time? And, if you've done that, just replace the actual function with the test and be done with it.

[jameshart]

One answer to that is to apply property based testing. If you're testing an addition algorithm, for example, you might test that

   A + B == B + A
   A + 0 == A
   (A + B) + C == A + (B + C)
   A + B != A + C (given B != C)

[slavik81]

It's a good idea in principle, though your third and fourth properties are not true of floating point addition.

When you have one available, testing against the results against a reference implementation is a little more direct. Though, given how easy those property tests are, you might as well do both.

[detaro]

Your comparison source could be running on a different platform, or be slower, or ... and thus not be a drop-in replacement. (E.g. in the example of a broken floating-point unit, compare the results with a software emulation)

[StavrosK]

Hmm, true, it may have been a different platform/slower, thanks. It can't have been hardware/software, as the GP said they were testing a library (which implies software).

[dagw]

How is the algorithm supposed to know what the right thing to return is

Some times I have, for example, matlab code that I 'know' is right and want to verify that my C/Python/Julia... code does the same thing. So then I just call the matlab code, call the new code and check if they're the same.

Or I have a slow exact method and want to see if my faster approximation always gives an answer 'close enough' to the correct answer.

[stephencanon]

With floating-point math libraries especially, it's frequently the case that it's relatively easy to compute the desired result accurately by using higher precision for internal computations; e.g. you might compare against the same function evaluated in double and then rounded to float.

This sounds pretty sketchy at first, but it turns out that almost all of the difficulty (and hence almost all of the bugs) are in getting the last few bits right, and the last few bits of a much-higher precision result don't matter when you're testing a lower-precision implementation, so combined with some basic sanity checks this is actually a very reasonable testing strategy (and widely used in industry).

[praptak]

If a function takes a single 32 bit argument you can run it on every possible value.

[detaro]

… this reply does nothing to answer the parent's question. It's obvious that you can run it on all possible values, but that doesn't answer the question of the return values. (unless you only want to verify "doesn't crash" or something similarly basic)

[praptak]

I believe the question was edited after I wrote my answer. This, or I just misread it - unfortunately HN doesn't let us know. Anyway, I'm providing another answer to the question as it is now.

[praptak]

Validating the answer is different than providing it:

   FLOAT_EQ(sqrt(x) * sqrt(x), x) && sqrt(x)>=0.0

is a pretty good test for the square root even though it doesn't compute it.