[spamizbad]

This person seems to be sharing anecdotes about the Data Science job market. That field, which exploded over the last decade, has experienced a rapid contraction. Data teams have shrunk and contain far fewer traditional “Data Scientists” and lean more heavily on engineers who orchestrate and plumb data around.

Data Scientists have become the “HTML programmers” of the tech bubble.

My advice: take a pay cut and pivot to becoming a business analyst or something.

[noirbot]

Yea, I feel like "Data Science" was always a Zero Percent Interest phenomenon. It's the software company version of Dogecoin or GameStop or AMC. If you've got free money, why not throw 10mil at the moonshot of discovering a diamond in the rough of your data that revolutionizes your business.

For the last 10 years, every data team I've been adjacent to has had a near infinite budget, hired more people than the rest of the company combined, had an abundance of "scientists" for whom any sort of data process, version control, scientific method, or cost controls were never part of the equation. It really doesn't surprise me that companies have decided they're not throwing good money after bad.

There's absolutely good work to be done in Data, but it requires people who know what they're doing, and not just "here's 10 years of postgresql databases. I dunno, find us 10% profit somewhere in it". There's not many companies that really have the data or personnel to make it happen.

[acdha]

I think that’s a bit harsh – but only so much, as I’ve had to explain basic stats literacy to several consultants – but there’s a common factor to many “AI” initiatives: some executives really want to be able to tell their buddies that they’re doing amazing things, and often believe there are huge untapped opportunities to make or save money which all of the employees they don’t trust haven’t mentioned.

In reality, of course, there probably just isn’t that much money at play so the high-budget approach is unlikely to break even, and more critically it’s unlikely that people are prepared to change how the organization makes decisions. I’ve known some people who worked for places where they had data showing solid improvements but it either hit politics or was simply small enough that their executives didn’t want to go with it because it wasn’t the magnitude they’d promised to justify the data science program.

[noirbot]

It's definitely harsh, and I want to be clear that I don't think it's innately a bad idea or destined to fail, but when it's easy to throw millions of dollars at a moon shot, trying to spin data straw into gold is worth a shot. It's similar to people trying to make a self-driving car, and how many of those are struggling/ending recently. It's not that there's nothing there, but when loans aren't free, you can't just throw money at it and hope.

It's a field that's just been over-hired because it's been a field where you can go "we're 2 years away from something big" for 10 years and the money was cheap enough to just keep paying people with questionable results or prospects. It doesn't shock me at all that people are having a lot of trouble getting work in the field. Even 8 years ago, going to college job fairs, I almost had to herd away Masters students in ML and Data Science who were already having a hard time getting a job and were resorting to applying for QA positions at web dev companies.

[ransom1538]

Eh. The real elephant in the room with "ML and Data Science" is checks and balances. They take over as the eyes of the company. Guess what department is seen as kicking ass? Guess which department is getting most money? Most hires? Data. Data.

It's like having the local police department check on the local police department.

[hn_throwaway_99]

What's worse about these anecdotes is that they ignore the obvious. I guarantee you some people are still getting hired for those open job reqs, it's just that when you have a big contraction and now have a ton more applicants for each open role, you're going to have many more anecdotes with people lamenting that "AI has broken the process" or "there are a ton of scams" or "insidious tech trends", because the real answer is just a lot more painful: there were tons of applicants for that job, and the person who was picked wasn't you. And (saying this from experience), that can just really, really suck, and when it goes on for a while it can be very difficult to come to terms with.