[torinmr]

A particularly hairy case is where you use variables as features which are themselves generated through other potentially-discriminatory processes. For example, including a "has committed a felony" feature seems like a no-brainer for a hiring or lending application, but now any racial or other discrimination present in the criminal justice system has now "infected" your hiring process, so that the outcomes of your hiring process are now racially biased even though your process itself was not.

[nickpeterson]

But at some level, doesn't that just mean reality is biased, and since we don't live in utopia, trying to model one in the small is a sandcastle?

[inimino]

What do you mean by "reality"? Society is pervasively biased, yes, that's the point. That makes it difficult to fight against that bias, but it is necessary if you want to live in a more equal world.

[dlss]

> What do you mean by "reality"?

The thing scientists (especially physicists) are trying to model.

[inimino]

You're not the person I was asking, and I didn't ask for a definition.

In case you missed my point, we aren't talking about physics here, but about society, which is a reality we create.

Appealing to "reality" in a discussion about bias amounts to throwing up your hands and dismissing the problem as just "the way things are".

It's like if you described a complicated social problem you've observed and I shrugged and said "physics, eh?".

[dlss]

> Appealing to "reality" in a discussion about bias amounts to throwing up your hands and dismissing the problem as just "the way things are".

Yes, that was how I read GGP's comment. I think you are overstating the degree of control that we have over society.

For example, I have often heard the case that society influences young women into roles that eventually prevent them from becoming engineers. Gendered children's toys are often used as an example of this. However, gendered toy preference exists before socialization has occurred (it has been demonstrated in 3-8 month old infants[1]). The same gendered toy preference that exists in humans has also been demonstrated in vervet monkeys[2] and rhesus monkeys[3].

[1] http://link.springer.com/article/10.1007/s10508-008-9430-1

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2643016/

[3] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2583786/

With those references now provided, I think it's safe to say gendered toy preference in humans is much bigger than simply a question of which toys we encourage children to buy -- there seems to be a considerable amount of genetics involved.

Hopefully now you are understanding what I meant when I defined reality for you. As I said I believe this is also what GGP was referring to.

So with that background out of the way:

> In case you missed my point, we aren't talking about physics here, but about society, which is a reality we create.

> It's like if you described a complicated social problem you've observed and I shrugged and said "physics, eh?".

A. How do you know it's a problem with the "reality we create" and not a problem with the reality we are stuck with? (Again, this is how I read GGP)

B. If we can trace the problem to something akin to male vervet monkeys preferring to play with Tonka trucks... what are we to do about it beyond ensuring that P(loan | race=1) = P(loan | race=2)?

I ask B because the idea of forcing my life choices onto someone else makes me feel ill. It reminds me of being forced to join the basketball team in high school, which I hated (though others seemed to love). The sick feeling compounds when I consider doing it purely on the basis of their race or gender in contexts where their race or gender is causal... and that's usually the course of action people on your side of this discussion recommend.

[inimino]

There is plenty of evidence that people have conscious and unconscious biases. There is ample evidence that the American criminal justice system, for example, is biased at several levels. I see where you're coming from: at any discussion of bias, you would rather blame it on genetics, which means we as a society are not responsible for doing anything about it.

I'm sure the people that have been forced into high-paying fields against their natural inclinations appreciate your feeling ill on their behalf, though. /s