[j16sdiz]

I am not sure what I should think of AI reinforced discrimination.

Some sensitive traits (e.g. Race) have high correlation with something we want to estimate (eg crime rate, credit score). The same traits can be correlated with thousands of different other attributes.

For example, to estimate the risk of loan default, (mathematically) i can use

a) race

b) zip code

c) 3 or 4 seemingly unrelated attributes, but still highly correlated to race

d) a few hundred attributes

e) a few million attributes, taking a PCA and trim down to a few hundred dimensions vector space

When does the discrimination begins or end? (a) is surely illegal, but you can argue (e) is still a proxy to the same thing.

There is no way to cut it fairly. It seems to me any kind of profiling should be illegal

[jauco]

Discrimination is just another word for “treating differently”. The discrimination that we generally disallow is the one where it relates to humans and where they are treated differently based on attributes they have no control over. That were either an accident of birth or faith (which is special cased as something you should not put pressure on).

When estinating a loan default, even of 99 people with a purple skin color default on a loan, the hundredth should not be expected to default on the loan just because of the skin color. Both because this is scientifically wrong (it’s not the skin color that causes them to default. There’s a confounding variable) and because it would put someone in a position that they can never get out of.

So the answer to your question is simple: you make a model where the attributes are causal factors for loan default. And you might need to special case attributes that are an accident of birth but that list is finite (listed in the law) and short and generally constructed to exclude strong causal variables.