[eanzenberg]

In my experience, there are just a lot of "bad" AI/ML engineers who don't fundamentally understand what data can do, what ML algorithms can handle, and how to piece it together to produce something of value to the end user. A couple of these people on a team can torpedo a project. Worse are those who sabotage projects or are general pain points of hindering progress. These may be jaded people who don't believe that ML has any value yet have titles like Data Scientist or ML engineer, and can bring team morale down. The economics are similar to a grad-school research project, yet is infiltrated by all sorts of people with 3 month certificates believing they are the star of the show.

The most important element of AI project success is the right people and the right team. Projects are long-term and failure can be often. It's not easy to succeed but cultivating the right people and their mindset is in my opinion a needle mover for AI projects, more-so than what data is available, what algorithms are tried, and what shiny framework people want to use.

[blueblisters]

There are people all across the system who fail have a poor understanding of how machine learning works, especially in the enterprise AI market. The most common hurdle I have seen is having to explain to salespeople and customers that models are not perfect - they often fail on known unknowns and unknown unknowns. No matter how simple to understand you try to make it using charts and lift curves, every model failure has a potential for customer escalation, costing hundreds of engineering and support hours and depressing margins.

[pc86]

Not trying to nit pick (but here we go), but allowing an escalation to cost hundreds of hours seems like a systemic corporate issue. It's absolutely a customer service hurdle, you can't just tell a customer sending in a ticket to shove it, and throwing something back them that they signed acknowledging it is never helpful. But even involving engineers in these cut-and-dry scenarios (whether the customer agrees or not) seems like a poor decision.

[jahewson]

This is equally true if you replace “ML engineer” with simply “engineer”.

[visarga]

Yes, I've come to the same conclusion. When interviewing ML engineers, I prefer to know they are exceptional programmers with passable knowledge of ML than the other way around. If they haven't learned to be good software engineers it's improbable they will in the future, but ML can be learned. In fact a ML team needs a large number of regular software engineers, there's a lot of non-ML code to work on, such as labelling interfaces, data pipelines and CI/CD for models.

[logicslave12]

Real ML and not just plug and play models takes a serious amount of knowledge

[RussianCow]

On the other hand, unless ML is your core competency as a business, plug and play models can get you really far.

[PaulHoule]

When you consider the chain from inputs to action you can go far with a small toolbox (e.g. logistic regresssion.)

Plug-and-Play models in visual recognition basically work at this point in time; in NLP they don't.

[eanzenberg]

I wouldn’t have ML engineers doing ML. They should be working on scaffolding, maintenance, production side, etc. The worst ML scientists (producing ML models) I’ve experienced were software developers who transitioned.

[visarga]

You just need one ML scientist for 4-5 software (or ML) engineers. If you wand to optimise time to delivery of products, you have much more to gain by improving the software engineering part, because regular SWE it's 90% of the product.

One of the main differences between ML in academia and industry is related to sourcing the training data. In academia they just use available pre-tagged datasets such as ImageNet, in industry you have to collect, clean up, organise, train, iterate with new data.

[MisterPea]

Yep, you pretty much hit the nail on the head. A few people in this thread are equating ML scientist's responsibilities to those of an ML engineer, however, they are very different.

Production development of a model is a very hard problem and it's interesting because I see few companies trying to tackle it. One of them is tecton.ai (heard about them on SED) and I'll be interested to see how they evolve their feature set because it still seems incomplete.

[PaulHoule]

Wow! This is the elephant in the room that doesn't get talked about in that article at all. You're also the only person to mention it, despite the fact that it is the only essential step to a commercial machine learning project.

[nimithryn]

Curious why you think the worst ML scientists were the former engineers (as someone who does a bit of both).

[jrib]

> ML can be learned

Curious: do you have a recommended pathway for that?

[Swizec]

Correct me if I’m wrong but isn’t this fundamentally true for any team working on any project?

[logicslave12]

No, there’s more ambiguity in machine learning projects. When you develop a website, aside from the design, it works or it doesn’t. Whether some kind of ml product can work at all is often team dependent

[cnasc]

> When you develop a website, aside from the design, it works or it doesn’t.

This really isn’t true. There are websites and web apps that “work” but are really suboptimal from a performance and UX perspective. It’s possible to do this right, but it’s much easier to do a poor job. You end up with something that kind of works and may even be profitable but which is a boat anchor around your company compared to a better approach.

[MisterPea]

Yeah not sure what they were getting at. There are also a lot of people who fundamentally don't know good software engineering practices and cause a ton of tech debt