[gwd]

> It seems much more likely that whatever method they use to do feature selection on this obviously high-dimensional data just happens to end up picking something that was a proxy for gender.

Yes, this is exactly what everyone thinks happened.

Nobody thinks people at Goldman-Sachs wrote

    if (applicant.gender == "F")
        limit /= 20

somewhere in their algorithm.

But regardless of how the thing happened, if millions of people are treated significantly differently for no reason other than their plumbing, that's a major problem. People have been talking about this "accidental proxy for gender" for years now; there's absolutely no excuse for no doing a basic sanity check to make sure that this kind of thing isn't happening.

edit: typo

[kyrieeschaton]

"for no other reason than their plumbing" is exactly what you just excluded in the prior sentence.