Hacker News new | ask | show | jobs
by squeaky-clean 3077 days ago
Old and somewhat contrived example, but the first thing to pop into my head is the famous fast inverse square root function.

    float FastInvSqrt(float x) {
      float xhalf = 0.5f * x;
      int i = *(int*)&x;         // evil floating point bit level hacking
      i = 0x5f3759df - (i >> 1);  // what the fuck?
      x = *(float*)&i;
      x = x*(1.5f-(xhalf*x*x));
      return x;
    }
I can't think of a way to write a test that sufficiently explains "gets within a certain error margin of the correct answer yet is much much faster than the naive way."

The only way to test an expected input/output pair is to run the input through that function. If you test that, you're just testing that the function never changes. What if the magic number changed several times during development, do you recalculate all the tests?

You could create the tests to be within a certain tolerance of the number. Well how do you stop a programmer from replacing it with

    return 1.0/sqrt(x);
And then complaining when the game now runs at less than 1 frame per second?

Here's a commented version of the same function from betterexplained.com.

    float InvSqrt(float x){
        float xhalf = 0.5f * x;
        int i = *(int*)&x;            // store floating-point bits in integer
        i = 0x5f3759df - (i >> 1);    // initial guess for Newton's method
        x = *(float*)&i;              // convert new bits into float
        x = x*(1.5f - xhalf*x*x);     // One round of Newton's method
        return x;
    }
It's still very magic looking to me, but now I get vaguely that it's based on Newton's method and what each line is doing if I needed to modify them.

I actually just found this article [0] where someone is trying to find the original author of that function, and no one on the Quake 3 team can remember who wrote it, or why it was slightly different than other versions of the FastInvSqrt they had written.

> which actually is doing a floating point computation in integer - it took a long time to figure out how and why this works, and I can't remember the details anymore

This made me chuckle. The person eventually tracked down as closest to having written the original thing had to rederive how the function works the first time, and can't remember exactly how it works now.

I think the answer is both tests and documentation. Sometimes you do need both. Sometimes you don't, but the person after you will.

[0] https://www.beyond3d.com/content/articles/8/

2 comments

For example, here's one way to write a test that sufficiently explains "gets within a certain error margin of the correct answer yet is much much faster than the naive way".

Using Ruby and its built-in minitest gem:

1. Write a test that does minitest assert_in_epsilon(x,y,e)

2. Write a minitest benchmark test that compares the speed of the fast function with the speed of the naive function.

Notice the big advantage for long term projects: if the hack ever ceases to work then you'll know immediately. This actually happens in practice, such as math hacks that use 32-bit bit shifts that started failing when chip architecture got wider.

> no one on the Quake 3 team can remember who wrote it

Exactly. We have the code file, but not any documentation separate from the code, such as notes, plans, attempts, reasoning, etc.

> Exactly. We have the code file, but not any documentation separate from the code, such as notes, plans, attempts, reasoning, etc.

I agree with you that it can be tested. But it doesn't explain anything about why it works, or what methods the author tried that didn't work as well. If you ever had to make it faster, or make it work better on different hardware, you'd be starting from scratch again.

> If you ever had to make it faster, or make it work better on different hardware, you'd be starting from scratch again.

Optimizations like these a great area for tests because the test files can keep all the various implementations and can benchmark them as you like.

This enables the tests to prove that the a new implementation is indeed optimal over all previous implementations, and continues to be optimal even when there are changes in external dependencies such as hardware, libraries, etc.

> But it doesn't explain anything about why it works

IMHO it does, for all the areas expressed in the original link and the parent comment.

Here are the original link examples:

1. "What [TDD] doesn’t do is get you to separate the code’s goals from its implementation details."

IMHO first write the code's goals as tests, then write the method implementations. The tests may need new kinds of instrumentation, benchmarking, test doubles, etc.

2. "[D]oes the description of what Module 3 does need to be written in terms of the description of what Module 1 does? The answer is: it depends."

IMHO write modules with separation of concerns, and with clear APIs that use API tests. If you want to integrate modules, then write integration tests.

3. "Quick! What does this line of code accomplish? return x >= 65;"

IMHO write code more akin to this pseudocode:

    return age >= US_RETIREMENT_AGE

    return letter >= ASCII_A
Write a property based test, ie generate a bunch of random inputs, then assert that all of them are within some (loose) margin of error.
> Write a property based test, ie generate a bunch of random inputs, then assert that all of them are within some (loose) margin of error.

So someone comes along later, look at your test, and wonders: why did you go through all that trouble? You can definitely write tests for a lot of that stuff, but they still don't fluently communicate the why of your choices.

This doesn't satisfy the time constraint though.

    return 1.0f / sqrt(x)
Passes a property based test but now your game doesn't actually run because it's much too slow of an operation on hardware at that time.

You can also test execution time too, but that's finicky and doesn't help explain how to fix it if you break that test (if there's no accompanying documentation).

But at this point all you're saying is "I can't think of a way of testing performance".
Performance can be tested in a unit test. You just need to measure the time needed to compute the function on a given set of numbers, then measure the time needed to compute 1.0f / sqrt(x) on the same set of numbers. The test succeed if your function is 10x faster. In future, the test may fail because sqrt has improved and this trick is no more needed.
No I'm saying a test for performance doesn't accurately describe the reasoning behind it without accompanying documentation.

Speed is the entire purpose of this method, not the numerical accuracy.