[taneq]

You've got it backwards, though. The algorithm isn't saying "this person is black and therefore shouldn't get bail". It's saying "this person shouldn't get bail (according to a calculated flight risk based on bunch of reasonable criteria)", and a disproportionate percentage of the people who are assessed as high flight risks just happen to be black.

[rebuilder]

And the dataset that this algorithm derives its predictions from is presumably a real-world dataset, i.e. one where black people form a disproportionately large portion of convicts and recidivists.

The point stands: using statistics to meter out justice IMO amounts to collective punishment. Of course that leaves the question if whether more conventional methods are any better, but now we're opening up a new can of worms, namely what is the goal of criminal justice systems and how should those goals be achieved...

[taneq]

So maybe, rather than sticking our collective head in the sand and saying "no, it's impossible that black people are more likely to break bail or reoffend", we say "holy crap that's obviously caused by something" and try to address the root cause?

You cannot fix a problem by pretending it doesn't exist.

[rebuilder]

There's a difference between saying a disproportionately large number of recidivists are black and saying black people are more likely to reoffend. The latter frames the issue in terms of "what black people are like", and worse, gives the impression any given black person is more of a criminal than any given white person.

I'm not usually such a stickler for language, but in this instance it does seem to me that this kind of profiling serves to perpetuate that narrative of criminality being a feature of a group of people, and doesn't help to get at the causes.