Hacker News new | ask | show | jobs
by Eridrus 2258 days ago
As somebody in an ML Engineering role, i.e. somebody who could be asked to either fix the logging infrastructure or build some models, I would have agreed with this.

But even in this day and age with ML being the new hotness, you will find people who are quite happy to work on infrastructure and don't have a huge amount of interest in training models themselves, and it is probably a lot easier to hire them than people who can do both, and you may get better results from actual specialists.

1 comments

I wrestle with this too, there's a lot of context to determine what skillset is better.

I suspect, if there are lots of relatively simple ML problems, then a generalist with integration chops will be more effective in getting them out quickly and "good enough". The specialist may take too long on models that are too heavy and impractical.

If there's one big ML problem (Google search, Netflix recommender, Amazon search, etc), where 1% additional makes a difference, then yes, specialist DS/modeler is probably preferred.

Larger, older org/heavier existing infra/more specialized culture will also tilt the scale towards specialists.

It's obviously a spectrum, but I feel like any org who is considering hiring a data scientist probably needs a data engineering team to begin with since you can do a lot of the analysis people want by just counting.

I also think it's unfair to specialists to say they will always overcomplicate things more than others, I've seen plenty of generalists with researcher envy do the same thing.