[bilbobagends]

This is well known. There is a paper from the FDIC that points out that if you make lending decisions based on platform (iPhone vs android or macOS vs windows) you are likely discriminating because iPhone and macOS skew older, richer, and whiter. It is one of the things that makes letting ML have access to big random data sets for lending and finance so dangerous.

[spoonjim]

If iOS predicts better loan repayment and it’s legal to use, why wouldn’t I?

[teruakohatu]

Assuming your question is not rhetorical: because it probably does not predict what you think it does.

People with higher incomes probably buy more expensive phones. You already have their income on a loan application, so you don't need a blackbox NN model to confirm it.

Throwing in a bunch of corrolated features into a model won't help improve accuracy.

But if you throw in a bunch of weak features that together sum to a predictor of race (or other protected attributes), then aside from being grossly unethical, may well be illegal.

[spoonjim]

If race and _____ are correlated, won't any predictor of _______ also be a predictor of race? That logic could be used to even ban using income as a predictor.

[teruakohatu]

Everything is corrolated [1]. So putting that aside, context matters.

Contextually there is no reason to suspect that a phone operating system has anything to do with anyone's ability to repay a loan, anymore than the color of their car or their eye color.

Income and existing debt does, obviously.

What you don't want, and what is not legal, is denying a loan because of nonsense predictors that happen to be very strongly corrolated to race and very weakly corrolated to the ability to repay a loan.

If color of car, or operating system, is a strong predictor then the model is probably being overtrained on the training data and probably wouldn't generalise well in the real world.

[1] https://www.gwern.net/Everything

[spoonjim]

I don't know, I think that people with red cars and people with white cars might have different loan repayment behaviors and I think the model should be allowed to try to figure out.

[runnerup]

The questions would be:

1) Will it continue to be legal?

2) Do I care about ethical concerns and does something being legal make it ethical?

Other people who are in a different position from you will be exploring this to determine if #1 should be changed.

[eh9]

If you want to racially discriminate, that’s on you.

[syspec]

You're not helping, even though you think you are.

You're just perpetuating the prejudice of minorities, even if your coming from a good place.