Hacker News new | ask | show | jobs
by ffssffss 1271 days ago
I think it's less a case of saturation and more a case of companies realizing that most data scientists don't actually deliver value commensurate to the salaries they were asking. Most companies simply don't have enough data, or don't have hygienic enough data, or don't have the engineering heft to build a reliable data pipeline, so the data scientists often find themselves set up to fail. I've seen that happen a few times.

But, it's not like the fundamentals of the field are wrong. Predictive modeling is still really useful. It's just larger firms are the only ones capable of realizing that value.
2 comments
GianFabien 1271 days ago
In my experience it is common to meet managers who think they have far more data than is really the case. What data they do have tends to be inconsistent due to changes in processes and program changes. This is particularly gruesome when a migration between ERP systems had been undertaken or after a company merger or even reorganization.

It is impossible for a data scientist to reliably identify the data problems of a prospective employer during the interview stage. The biggest companies often have the messiest data. But you would need to talk to the IT staff in the trenches to discover that fact.

link
FridgeSeal 1270 days ago
> In my experience it is common to meet managers who think they have far more data than is really the case. What data they do have tends to be inconsistent…

Can also back this up 100%.

Too many times I’ve had “we have some x data, we need you to do y”. Turns out, the data they “have” is a 15 line excel file of unknown provenance and replicability; nobody in the dev/marketing/finance/etc team has the time, interest, or project alignment to care what you want, and what they’re trying to do needs a team of phd’s anyways, but all of this falls on deaf ears.

link
boredemployee 1271 days ago
Well, where I work at, I was hired as a data scientist and end up doing everything related to data lol, engineering, analytics and what not. It doesn't matter as long as they pay me to do that.
link
FridgeSeal 1270 days ago
Same here!

I ended up shifting more towards the dev side - devops, software engineering, etc.

link