Hacker News new | ask | show | jobs
by eeegnu 1698 days ago
Looking through the source where the tests are conducted,

for (double x = 1; x < 1000000000; x += 0.5) {

      counter++;
      int64_t tmp;
      if (to_int64_simple(x, &tmp)) {
        sum += tmp;
      }
}

The fractional part is always mostly 0.5, or 0.10000... While that shouldn't really affect things, I think a proper test would sample randomly, though this runs into the problem of the random number generator itself influencing the time spent, so I can see the appeal for this method. Perhaps generating a random float array with N elements and calculating the results for x + arr[x%N] would accomplish this.

2 comments

If your goal is to benchmark something like to_int64, you should be benchmarking it on a wide range of values instead of just ones and one-halves. You can always pre-generate a diverse data set and shove it in memory - how expensive that is doesn't matter, since all your benchmarks will be paying the same cost to read the test data (unless one of your benchmark test cases is so profoundly slow that it pushes the test data out of cache)

Similarly you definitely should have a table of known numbers you want to test and not just go through the space by 0.5. You want to test denormals, stuff like 1/3, etc.

Unit test cases should be explicit. For determining if his algorithm is correct he could implement it 2 completely different ways and do bitwise fuzz testing to try and find cases where the output differs.
Or (if he did it with floats), tested it with all 4 billion floats. [1]

[1]: https://randomascii.wordpress.com/2014/01/27/theres-only-fou...