Hacker News new | ask | show | jobs
by bicubic 3341 days ago
In reasonably sized datasets, you'll typically find a lot of interesting information and relationships that are only loosely or not at all related to what the analyst is actually paid to do at the time.

Analysts who only find the specific thing and end their work on that are a dime a dozen, and need to be micromanaged. Good analysts will find all the other interesting stuff on their own and inform the business about it. Those good analysts are the explorers, and banning those people form exploring during training seems like an effective way to take talented budding analysts and turn them into mediocre ones.

1 comments

In reasonably sized datasets, you'll also find a lot of spurious correlations simply by chance. That's one reason in science you're supposed to write down your hypothesis and methods of analyzing data before touching the data. Otherwise you risk finding some random noise and thinking it's important.