[elfakyn]

That gives rise to a very interesting concept: ML-based bias assessments. If you take some real-life hiring data (or other applications such as sentencing or generally human behavior data) and train the AI on it, then run it through a bunch of tests to see whether there's bias, that can reveal trends in the underlying training data.

I can't imagine this not already being a thing, but I haven't really heard of people using this method.

[bloak]

I don't think you can detect "bias" by running "a bunch of tests". "Bias" is a very slippery concept and is probably essentially subjective. When people say an algorithm is "biased" what they seem to mean is that when the judgements of the algorithm are compared with the judgements of a committee of fair-minded and diligent humans then the number of positive outcomes for members of some fashionable minority that we care about is less that what it was with the human judges. It's hard to automate that. And in any case, if you manipulate the algorithm until it "passes" a test like that then you might not really have improved it: when you turn a measure into a target it ceases to be a good measure.