[yanis_t]

Simon, is your pelican test really captures differences among models or should you at least try like 10 times or something to average the random effects

[simonw]

I've been meaning to do a "run 3 times and pick the best" version for quite a while, I should really pull the trigger on that one. Currently it's one-shot only.

[notaharvardmba]

You could run 3 times and overlay/average the images to show how consistent they are

[xiphias2]

Best-of-3 would be cheating, ruin the test, middle of 3 makes more sense

[nik736]

Why would you need the 3rd run if you pick the "one in the middle"?

[jmaw]

Middle as in not the best, and not the worst. As opposed to the second generated in sequence.

But not the best/not the worst is somewhat subjective.. so not sure how well that would work.

[BrokenCogs]

I think GP meant picking the median pelican