[pessimizer]

Because both the original conviction and any recidivism is determined through the decision-making of people who are aware of race and racial stereotypes. The AI would just be laundering the decisions you were already making, not improving them.

edit: imagine I was a teacher who systematically scored people with certain physical characteristics 10% lower than people who didn't have them. Let's say, for example, that I was a stand-up comedy teacher that wasn't amused by women.

If I used an AI trained on that data to choose future admissions (assuming plentiful applicants), I would end up with an all-male class. If this happened throughout the industry (especially noting that the all-male enrollment that I have would supply the teachers of the future), stand-up comedy would simply become a thing that women were seen as not having the aptitude to do, although nobody explicitly ever meant to sabotage women, just to direct them into something that they would have a better chance to succeed in.