[_qbxp]

Sort of tangential, but I felt this exact same way when I moved from being a post-doc in neuroscience to being a data scientist. The impostor syndrome was so strong it was painful. It has subsided now a bit because I know I'm able to bring value to my company - despite the fact that I know there are much more capable and qualified data scientists (by a large margin) out there, and despite the fact that by-and-large 'ML' and 'AI' is definitely a buzzword around here. But it really, really motivates me to strengthen where I'm lacking.

The funny thing is, it took me about a year to find this position after a good amount of rejections. About a year after I got my data scientist title, I've been contacted by recruiters from places I would have never expected to be contacted from (Amazon, Microsoft, FB, etc.). Did a few interviews, and realized during those interviews that I still have a lot to learn.

For one of the interviews, they gave me a take home assignment where they literally duplicated a column in the feature matrix... I didn't catch it, and during the phone part of the interview I get asked 'do you know notice something interesting about those two feature distribution plots you have there?'

"Hrmm, no I don't. Oh, wait, they look pretty similar."

"They're exactly the same."

"... shit."

[gaius]

they literally duplicated a column in the feature matrix

This is called colinearity - you can check for it by comparing the rank of the matrix to its number of columns. In R qr(X)$rank. Good to add this to your EDA workflow.