[zachwooddoughty]

>> ProPublica labelled the algorithm as biased based primarily on the fact that it (correctly) labelled blacks as more likely than whites to re-offend (without using race as part of the predictor), and that blacks and whites have different false positive rates. >> In the conception of these authors, “bias” refers to an algorithm providing correct predictions that simply fail to reflect the reality the authors wish existed.

The gist of the article is that statistical bias is not the bias that journalists are interested in. The article doesn't discuss how these biases are related or relevant to each other, but rather assumes that statistical bias is the only one that should matter. I think the article is missing a discussion of the gaps between what are the measured inputs into your statistical model, and what can be acted upon from a policy perspective.

As a thought experiment, suppose the only two inputs that determine a person's recidivism rate is their past criminal history and whether they had lead poisoning as a child, but that of these two we can only measure past criminal history as an input into our algorithm. If race is strongly associated with childhood lead poisoning (such as in real life [1]), then our algorithm might get higher classification accuracy by including race [2] as an input in its training data. This might have less statistical bias, but would bias against individuals of a race who are in truth not at higher risk of recidivism.

[1] https://scholar.harvard.edu/files/alixwinter/files/sampson_w...

[2] The actual COMPAS algorithm doesn't use race as an explicit input, but that doesn't really change the issue.

[zo7]

I feel the author failed to address the crux of why people argue that "non-statistical" bias is bad – that we should be judged by our actions, and not by factors out of our control such as race, class, the family we were born into, or where we were born.

If we included every aspect about a person in some statistical model, we may discover "uncomfortable truths" that hold true for the general population. But these truths, while statistically correct, may fail our test for what we consider to be philosophically fair, and ultimately undermine an individual's agency to act independently.

So perhaps in your experiment, the problem is that our feature selection is not reflective of the values we'd like to uphold, and that aspects like "had lead poisoning as a child" is not a sound feature to include in our model because it measures aspects of a person outside their control. Instead maybe our feature set should only include aspects that measure facets that are under the individual's control such as community service, whether they still associate with other criminals, whether they have or are pursuing education, whether they have children to care for, etc. (or some other feature set that's more thought out and sound, but you get the gist)

This still may not have as good accuracy as a model that included other features about the person, but it's arguable that this system would be more fair, especially over a model using more features but was artificially fudged to satisfy some prior about what we consider fair/unbiased.

[bobcostas55]

>I feel the author failed to address the crux of why people argue that "non-statistical" bias is bad – that we should be judged by our actions, and not by factors out of our control such as race, class, the family we were born into, or where we were born.

This is exactly what the author is talking about. You are comparing the predictions against your fantasy of a world where these aspects do not matter because they're not "fair". When they don't match up, you call the predictions biased. But these factors outside our control do matter, accounting for them does not introduce bias, and averting our eyes will not change that fact.

[zo7]

I'm not claiming that we should ignore these aspects and lie to ourselves about reality, in fact I'm acknowledging that these relationships exist. My greater point is that the author is trying to use the meaning of statistical bias to dismiss what journalists/laypersons consider bias without addressing why the latter is concerned with bias in the first place.

My suggestion is that we should be using a better feature set that only looks at aspects that we can reasonably hold an individual responsible for rather than using demographic information which is out of an individual's control. If we have two convicted criminals with similar crimes, behaviors, and histories, but one is white and grew up in a wealthy neighborhood while the other is black and grew up in a poorer town, why should the former be granted a higher probability of parole than the latter? Why should either of them be held responsible for the actions of others? Even if in expectation people from the latter demographic were more likely to reoffend than the former, that is not justice – it undermines liberty.

[phiskk]

If "these truths, while statistically correct, may fail our test for what we consider to be philosophically fair" then perhaps it is your philosophy of 'fairness' that is biased and incorrect.

[zo7]

Would you like to point out how it's biased and incorrect? My latter claim is that we should base features on aspects that an individual could be reasonably held responsible for, rather than aspects out of their control such as their race, gender, or where they grew up.

So my philosophy of "fairness" here is that we should hold people responsible only for their own actions, which, while off-the-cuff, seems like it would agree with most justice systems. If we included demographic information in our models, we would effectively be holding individuals responsible for the actions of others, which doesn't seem sound.

[bobcostas55]

>This might have less statistical bias, but would bias against individuals of a race who are in truth not at higher risk of recidivism.

This is true of all predictive variables and all predictive models. It doesn't justify the recent panic aimed at ML at all.

[marchenko]

Journalists may not be using the term bias in a statistically appropriate way, but they do seem to be capturing a colloquial sense of what bothers people about these models: the potential for the individual to be subsumed into his demographic, and the use of suspect classifications (like race in the US) as highly salient but 'silent' factors in the model. I don't think this is a misleading use of the term for the average reader, whom I suspect would consider the FICO redlining example as exhibiting 'bias' as the term is commonly understood by laypersons. The fact that these models are difficult to understand and interrogate for the average citizen is not exactly a point in their favor, since a lot of consensual governance is based on transparency and information symmetry, even at the expense of optimization/efficiency.

[zachwooddoughty]

Of course all models are biased, but we can try to build models that minimize the types of bias we care most about.

I think the so-called panic comes from the fact that more complicated algorithms are difficult for an untrained person to understand. For example, it's also illegal to discriminate housing/lending based on race, but if I hide that discrimination inside of a sufficiently-complicated ML model, I may be able to get away with it.