|
|
|
|
|
|
by NutriSugar
3037 days ago
|
|
|
The systems also encode in existing discrimination. For example, women are less likely to be investigated for a crime, charged with a crime, convicted of a crime, and receive lower sentences when convicted. In some cases (especially involving sex crimes) there is explicit sex based bias written into the law itself. This means a lot of data on who commits crimes has a strong sexist bias, so any automated algorithms have a strong possibility of reinforcing the existing bias. The same will happen for race, class, and many other factors. There are also cases where society wants an incorrect bias put into place. For example, parole risk assessment software vastly underrates the threat of certain classes of criminals compared to what society and police think it should because there are major myths about rehabilitation and recidivism that are as popular as they are wrong. Perhaps the worst part is a total lack of transparency in the existing algorithms that determine risk. With enough data you can reverse engineer it, but that doesn't give the same impact as seeing the rules themselves. For example, one parole group I developed software for had a risk rating that appeared to automatically rate women a risk level lower for the same crime. Imagine if the same was done based on race. |
|
|
Well, the same is done based on race, they just don't need to write that into the software.
The point I'm trying to make is that it's going to be impossible to eliminate all bias in the system, but people are justified in their aversion to systems that automate bias into our system. Palantir certainly has the potential to do so, and the total lack of transparency definitely doesn't agitate in its favor.