[opportune]

>Maybe that's the problem. In lots of AI/ML problems you just CAN'T know ahead of time what can be deliver you need to spend the time and resources to do it and then see how well it works...

Sure, this is true in some cases, but in most cases I think anybody with a fundamental understanding of ML could tell you yes/no by understanding or estimating whether your input data contains enough signal to explain your desired output. In many cases where ML "fails" it's relatively obvious that there is not enough signal to cover the output; in many other cases you can be quite assured a priori that ML will give you a decent solution without needing to test it first.

There will of course be issues if your practitioners are unable to distinguish these cases or if they are structurally not empowered to say so (e.g. if they are just ordered to make a model using data X to do Y). That is probably what happens in a lot of cases when business leaders blindly decide to "add ML" to their business.

The fact of the matter is that ML is very good at solving problems where an input signal strongly correlates with the output signal, and it's appropriate over other approaches when the mapping is hard to define, such as in recognizing images of birds. If you can apply some transformation to your business problem into something with this structure, you can apply ML; otherwise you probably shouldn't.