[mproy]

" ... other correlated factors will allow the algorithm to continue to enforce this racism"

Which correlated factors are you thinking of?

[advisedwang]

Some proxies for race: - Where somebody lives - Their name - What sports teams somebody supports - Their finances - consumer preferences - education level - what medical conditions somebody has - political leanings

I'm not saying these factors can't legitimately be used in a risk assessment, but they could be used to make a good bet on race.

[Sacho]

Finances? While it is true that blacks and native americans have the highest poverty rates, if you take a random poor person they are about 50% more likely to be white than black. Same with the many of your other "proxies". Therefore they would be a poor bet for race.

If you were using these proxies to identify the race of a person jailed for a crime, they might be good predictors, but only marginally more useful than just blindly guessing black, since they constitute the majority of incarcerated people.

I think, by attempting to remove the smoke screens a "real racist" would use, you give them a new one - "look, our opponents want to ignore actual data".

[mproy]

I don't see how those factors are relevant at all, regardless of the race of the criminal. The problem to address is the lack of transparency. Trying to guess what inputs are being factored in is pointless.

[vmarsy]

Even if you are careful to remove all features you think could tell the race (let's say geographic location etc), it could still be giving harsher sentences to minorities if those are treated differently upfront. For instance, it would make sense for the model to learn that the more charges count the harsher the sentence. This sounds right: a bank robbery should be less punished than a bank robbery + a carjacking, at least in the US judicial system.

But now let's say minorities gets a "resiting arrest" charge on top of their original charge more often, because of many factors such as the police bias, the bias of the minorities towards police, etc. (By bias here I mean all spectrum : racism, but also fear of the police etc).

As long as minorities are treated differently upfront, then the model would treat them differently.

[mproy]

But if all of that is true, the model doesn't change the equation. The problems you outline need to be addressed upstream. Trying to account for and correct them at the point of sentencing is very problematic.