| Don't be surprised if those predictions are heavily biased against minorities and poor people. Do you care if they do? It's a similar problem to using ML to give people credit scores. If the training data includes a lot of minorities and poor people breaking laws / delinquent payments, then your ML will simply key on race/economic status as a predictor. So you've built a system that simply targets those groups. But you might object and say that this race/economic status targeting gives the highest accuracy! It was only learned in the training data, after all. You can make a great classifier that is extremely unfair. So you have to realize there is a conflict here between accuracy and fairness. This means there is a conflict between observational data (training), and using that data to produce decisions/outcomes. If you make decisions/outcomes that reinforce the training data, you do not give racial groups/low economic status people a chance to improve their lives. That is extremely inhuman, predatory, and unfair. |