[jerf]

So you know, I've gone ahead and submitted that: https://news.ycombinator.com/item?id=13158883

I do think it's important for people to be more clear about the goals they have for these systems. Do you want them to be accurate regardless of anything else, or do you want them to accomplish particular social goals by deliberately bending the results? There's nothing necessarily wrong with the latter, though you will inevitably find once you're being clear about the latter that even people who are generally ideologically aligned with each other will discover they have different and mutually conflicting goals...

Anyhow, either way, until you have a clear specification about goals you can't determine whether you're accomplishing them. A vague goal of being "nondiscriminatory" doesn't cut it for computers... "nondiscriminatory" is a set of possibilities, not a unique specification. (And if you disagree about my claim it's a set, just try to sit down with one of the aforementioned ideologically-aligned people and try to hammer it out to a coding level of detail.)

[yummyfajitas]

Not only that, there are impossibility theorems that say you can't meet everyone's definition of "nondiscriminatory" except in a few trivial cases.

https://arxiv.org/pdf/1609.05807v1.pdf

I.e. every algorithm will be "discriminatory" unless you predictor is perfect (i.e. gets 100% of decisions right). It's just a question of which kind of discriminatory they are. The same is true of any human decision process.

Unfortunately this means that we will be subjected to a slew of innumerate reporters posting "XXX is discriminatory" articles, regardless of how decisions are made.