I wonder how an algorithm with low explainability will be treated, like a large nueral network. Seems like it would be hard to pin it on the algorithm rather than the data.
If there's a way to objectively evaluate quality of output, that seems sufficient. It's not like a building inspection where it's a component-based evaluation. The results need to be reasonably and verifiably reliable to an appropriate degree, but the courts don't need to care how that is accomplished.
> If there's a way to objectively evaluate quality of output, that seems sufficient.
But that's the whole problem. If you had something that calculates the the thing algorithm is supposed to do in a better way than the algorithm actually does it, you could just use that as the algorithm.