|
|
|
|
|
|
by omegaworks
2379 days ago
|
|
|
The argument is that upstream often doesn't have (or is willfully blind to having) a clear understanding of the features impacting the results of the model. The author is making the case that model builders need to be aware that their models may further structural racism, and need to do the work to push this awareness upstream. Upstream may think the model--which to them looks like a black box--is perfectly rational, optimally profitable and socially beneficial in a way that is it is not. We have numerous examples where a computer has driven a decision and humans carried out its orders in ways that harmed people. Remember the man violently dragged off the United Airlines flight in 2017[1]? ICE justifies detention by tweaking their risk management software to always recommend detention[2]. This is why we need to care as people building the systems that make these decisions. 1. https://www.nbcnews.com/storyline/airplane-mode/united-fiasc... 2. https://www.vice.com/en_us/article/evk3kw/ice-modified-its-r... |
|
|