Hacker News new | ask | show | jobs
by jaredtn 2138 days ago
What do you mean by p-hacking? In statistics that refers to running a large number of experiments and taking the one with the best results, but ML trains for hundreds of thousands of steps on a single model.
1 comments

ML runs hundreds of thousands of experiments, each on a slightly different model, and takes the one with the best results.
That's not necessarily p-hacking. It becomes p-hacking when the model is hyperoptimized to the test set and thus fails when applied to new data from outside the test set.