[meheleventyone]

The problem is that AI and more generally 'algorithms' are or were presented as neutral and unbiased. As such their biased results prop up a biased system.

I don't think people are against using ML and for biased human systems. Just pointing out the ignorant, naive and lazy deference to computers that often occurs in human systems that share the same bias.

In short I'd think most people who are against biased AI are also against biased human systems for very similar reasons.

[DuskStar]

Of course, sometimes reality is also biased, and the AI systems are just accurately reflecting reality. And that's an even bigger elephant.

[meheleventyone]

I’m not sure what that even means if we know we can bias outcomes. Pretending there is some kind of natural state that is for the sake of being natural preferred seems odd given humans propensity to change the world to suit. I also suspect for many that ‘reality’ is really just a dog whistle for their preferred biases. Not to mention the entire issue with deriving and ought from an is.

[wongarsu]

Suppose you train an AI to predict how good people are at weight lifting, trained from a bunch of seemingly unrelated data (maybe you want to hire bouncers or construction workers). You will find that the model predicts better performance for males. You notice this, identify that men are more likely to go to the gym than wimen, and modify your data to compensate for this. But when you rerun the model men still show better results. You find some other biases in your data. You find societal biases, like role models for girls not being physically strong. You even take some women and show that with training they outperform average men.

You can modify reality, but our understanding of biology - especially hormones - clearly tells us that the AI was right: men are generally better than women at weight lifting.

I'm not saying that every issue is like that, but it would be foolish to ignore that sometimes reality is biased, sometimes in obvious ways and sometimes more subtly.

[meheleventyone]

What I was getting at is that our important choices are about outcomes and those have nothing really to do with assumptions about reality. For example all should be equal before the law. A statement that is supposed to be true but very obviously isn’t.

Your post is great for the assumptions it encodes. Like what does it mean to be good at weight lifting? And that for some reason being good at weight lifting is a good proxy for being a good bouncer or construction worker?

For an off the cuff example it’s a great way to demonstrate the sort of bias we can naively introduce then defend because it’s just ‘reality’. When really it’s much more complex than identifying a relevant trait and assuming everything else falls out of it.

[Chris2048]

> Like what does it mean to be good at weight lifting? And that for some reason being good at weight lifting is a good proxy for being a good bouncer or construction worker?

being a good weight lifter, means you can lift heavier weights then a less-good weight lifter. Whether this is a proxy for anything isn't relevant, because it's a purely contrived example. There are clearly jobs where physical strength (among other things) is important, and given the context of this example, there is no guarantee that a more complicated model evens out the differences.

The point of the example is, basically "there are some things which might discriminate strongly on the basis on physical traits, which might end up correlating with race/sex etc" - ask for a better model by all means, but there is no guarantee the perfect model will never correlate strongly with some political demographic, and hence be controversial.

[meheleventyone]

Those were rhetorical questions I fully understand what the point of the example was.