[apohn]

>So ML/DS people need to overpromise in order to get anything approved, otherwise they'd just have to sit around and do nothing, and overall everyone would be worse too, bc that real but unreliable progress would never happen.

IME with 8+ years as a Data Scientist in internal and external consulting roles. What I like to do is propose a phase 1, phase 2, phase 3, etc. Phase 1 is almost always a mix of scoping and "Does it even make sense to do this."

Only a fraction of the teams/customers I work like appreciate and have a tolerance for this longer term strategy. Most of those projects are a success and people get value out of using a model in addressing their business problem.

Most people hear this and respond with "No, Phase 1 needs to solve our poorly defined business problem and show massive value in 3 weeks so we can boast about how innovative we are being. You need to AI this problem. AI it now. Do some of that AI stuff. Give me the AI." Most of these fail and I know they are doing to fail, even if any models turn out be useful.

I will say that there there Data Scientist/AI/ML people who really just throw models at stuff and don't think about the value of what they are trying to deliver, but a lot of them are typically inexperienced and just have new toy syndrome. It's not really any different than somebody who reads a bunch of books (e.g. leadership, business) and then gets overly excited about using what they learned. They'll grow out of it or leave the profession when reality hits.

[hectormalot]

Agree. We do something similar:

* 1 month (or less) to prove that it makes sense to continue. We do start out with a pretty well defined problem definition though. Outcome is typically a bunch of jupyter notebooks proving that we get at least some predictive value that helps solve the problem. If no proof: no-go

* 2 months to upgrade it to something that works well enough to run a real pilot (e.g. shadow run, run with a few agents, in an A/B test, etc.). If no success: no-go

* Then hopefully in production within another 3 months. We're in a pretty regulated industry, most of this time is _not_ engineering.

The phases help set priorities as well. Doesn't always make sense to spend (much) more time on scalability if we don't know it works yet.

[apohn]

>We do start out with a pretty well defined problem definition though.

One of the things that really helps this happen is when the management structure (on both sides) understands a defined problem statement that both sides agree to is a requirement for a project to have any hope of success.

The above isn't specific to Data Science projects. It's just having a management structure who knows the right time to support things being pushed back.