Hacker News new | ask | show | jobs
by natalyarostova 3084 days ago
As a data scientist I have wondered if this field is particularly suited to imposter syndrome. My formal background is economics, and every once in a while I become terrified at how little formal statistics I've studied, or large gaps in data structures etc. although I'm similarly surprised at how far I've gone by just going home and studying the basics when I run into something I don't know, and the gaps in knowledge some coworkers have in areas where I know more.

...although I have met a few genius data scientists who seemingly really can do everything. Although I'm pretty sure they are paid upwards of 300k.

2 comments

I'm pretty certain data science is worse than normal fields. It's probably due to the fact that huge proportion do have PhDs, but that a PhD is not required for the vast vast vast majority of what we do.

So for instance, I totally feel like a data science imposter, but in the last year have done the following: - Pushed a custom deep learning NLP model to production - Created and maintain company's ETL and data warehouse mechanisms - Performed statistical analyses to find ways to better target and increase customer engagement. - Implemented event tracking and performance metrics across products - A sales prediction product that has contributed to $~5M in incremental revenue

Somebody obviously believes in me since I've grown the team from just myself to ~6, but I also know that I've had dozens of past colleagues that would instantly disqualify me since I 1) don't have a PhD and/or 2) can't/don't read statistics/machine learning papers

I tend to agree. I think it's because the field uses statistics, which most people have decided are incomprehensible and don't even try to understand. Combined with how bad our brains are at thinking statistically, and you have a powerful desire-for-avoidance by non statisticians.

Combine with the value that good ones can provide, and you have a perfect situation where a boss or peer just thinks thar be dragons in the work sphere of the statistician.

I think the issue lies in the fact that data science is a massive umbrella term which really covers multiple large fields, namely statistics and computer science, both of which by themselves are massive. Its very difficult if not impossible for people to feel like they have enough expertise in all of the subjects that fall under the umbrella of "data science". Which typically leads to lots of self-teaching, learning on the fly, and hacking solutions together, which is often the source of all impostor syndrome feelings.