Hacker News new | ask | show | jobs
by rwmj 862 days ago
Sure, but let's say that we test this and it is predictive on new data (not overfitting), but we have no idea at all how it works. It's still a useful test.
1 comments

The retrospective regression on a specific dataset might discover a true correlated quantity, if any true correlated quantities were there and their signal was more prominent than the combinations you get from the noise. However, this analysis will always discover a quantity that correlates, by design. These retrospective studies can prompt prospective studies for a correlated quantity (a biomarker in this case) and the careful analysis of the retrospective study methodologies and results can suggest the design of such prospective studies; if a prospective study works, then that is fantastic. The retrospective studies are mostly there for statisticians to figure things out for future tests, except when the signal is simple and phenomenal.