Hacker News new | ask | show | jobs
by r-zip 1301 days ago
I think a big part of the problem is that most PMs are non-technical, and at some point up the chain so are most managers. Data science, when done thoughtfully, requires you to appreciate minute issues from data collection and management all the way through feature engineering, model selection/design, and validation.

The biggest, hardest bridge to cross was an appreciation of the importance of metrics. For some reason, getting a business person to grok something as simple as precision/recall/F-beta is a near-impossible task. You can do multiple presentations on it (after having honed those presentations over years with multiple manager audiences), and it never sticks. It's always "what's the accuracy?" It's impossible to do good work when your bosses insist on measuring and therefore optimizing for the wrong thing (which in my experience consulting for multiple Fortune 500 businesses, they always do).

Even worse, many organizations have such broken politics/cultures that the managers can't even tell you the big picture of what the project is trying to accomplish. Once you finally piece it together from the people who know their roles in-depth, it becomes clear that what they're trying to do is totally infeasible. At least that was my experience more than half the time.