Hacker News new | ask | show | jobs
by j_not_j 396 days ago
Some comments, in random order:

- your test codes are not reproducible: running twice generates different sets of numbers because they have unknown seeds. As a result, if you change (a) compilers or compiler switches, (b) operating system versions, (c) host processors or (d) architectures, the question arises: what is wrong? What is different? This is known as a regression test.

- try shishua as another speedy PRNG. See https://espadrine.github.io/blog/posts/shishua-the-fastest-p...

- You only test a few times? I think one hundred BigCrush tests, using a set of 100 seeds, would be suitable. Takes a few days on an RPI 4 (with cooler). Run the same 100 tests on a Ryzen and Xeon, just to be sure. They should be bit-for-bit identical.

- 100 BigCrush tests should show only a handful (4 or fewer) duplicate test failures.

- your seeds are almost great: too many people think "42" is random in a space of 0 through 2^64. But 0xdeadbeef is so 1990s...

- you don't need different seeds per PRNG; you can generate reproducible ones (2x to 4x 64bit) from a single good 64-bit seed and your favourite PRNG. Your test code should read a seed or set from the command line (see first item).

- warmups? Really?

- Remember that BigCrush and other tests are created by mathematical people not practical people. Do they test for equal numbers of odd and even results? Hmmmm....

- try Collatz, see https://news.ycombinator.com/item?id=39733685

- the tests are very cache-friendly; nobody does this. It's true that everybody compares against this unrealistic scenario, however.

- if you've spent a month in twisty little PRNG passages, all alike, you are on the first steps of your journey. Take a towel.

J

1 comments

What a great set of feedback. Thank you! I'll look at it as an acton item list.

And you are so right about being just one month in. Every time I think I'm starting to understand what's going on here, I realize the maze just keeps getting deeper.