[teruakohatu]

Assuming your question is not rhetorical: because it probably does not predict what you think it does.

People with higher incomes probably buy more expensive phones. You already have their income on a loan application, so you don't need a blackbox NN model to confirm it.

Throwing in a bunch of corrolated features into a model won't help improve accuracy.

But if you throw in a bunch of weak features that together sum to a predictor of race (or other protected attributes), then aside from being grossly unethical, may well be illegal.

[spoonjim]

If race and _____ are correlated, won't any predictor of _______ also be a predictor of race? That logic could be used to even ban using income as a predictor.

[teruakohatu]

Everything is corrolated [1]. So putting that aside, context matters.

Contextually there is no reason to suspect that a phone operating system has anything to do with anyone's ability to repay a loan, anymore than the color of their car or their eye color.

Income and existing debt does, obviously.

What you don't want, and what is not legal, is denying a loan because of nonsense predictors that happen to be very strongly corrolated to race and very weakly corrolated to the ability to repay a loan.

If color of car, or operating system, is a strong predictor then the model is probably being overtrained on the training data and probably wouldn't generalise well in the real world.

[1] https://www.gwern.net/Everything

[spoonjim]

I don't know, I think that people with red cars and people with white cars might have different loan repayment behaviors and I think the model should be allowed to try to figure out.