I do think it's having some impact now. Google, for example, did not used to use ML for ranking sites. So they had a reasonably good handle on why sites would end up ranking where they did.
They then introduced several different ML add ons, like Vince, Panda, Penguin, etc. They know roughly what the algorithms are doing. But, as I understand it, it's still opaque enough that they can't explain exactly why a specific set of results is what it is. Especially since they don't all work together. They run one after the other, like a pipeline.
That is, there are some amount of false positives or negatives they can't explain. If you report something, they can guess and experiment around a bit.
For some spaces, they are basically king makers or breakers. So ML decisions are driving real world consequences. To the degree, though, that the results are "good enough", it won't ever get looked into. The acceptable error rate is the one that doesn't affect Google too much. An actual flaw could go unnoticed for quite some time.
They then introduced several different ML add ons, like Vince, Panda, Penguin, etc. They know roughly what the algorithms are doing. But, as I understand it, it's still opaque enough that they can't explain exactly why a specific set of results is what it is. Especially since they don't all work together. They run one after the other, like a pipeline.
That is, there are some amount of false positives or negatives they can't explain. If you report something, they can guess and experiment around a bit.
For some spaces, they are basically king makers or breakers. So ML decisions are driving real world consequences. To the degree, though, that the results are "good enough", it won't ever get looked into. The acceptable error rate is the one that doesn't affect Google too much. An actual flaw could go unnoticed for quite some time.