[tripletao]

The linked article already notes that model accuracy degraded after their reweighting, ultimately contributing to their abandonment of the project. (For completeness, they could also have considered nationality in the opposite direction, improving accuracy vs. nominally blind baseline at the cost of yet more disparate false positives; but that's so politically unacceptable that it's not even mentioned.)

My point is that even if we're willing to trade accuracy for "fairness", it's not possible for any classifier to satisfy both those definitions of fairness. By returning to human judgment they've obfuscated that problem but not solved it.

[tanewishly]

My point was that there is no test (or classifier) that can always guarantee that one definition of fairness by itself, irrespective of the base rate. If the classifier acts the same independent of base rate, there are always base rates (ie occurrence rates in the rates population) for which the classifier will fail the given definition.

That illustrates that the given definition cannot hold universally, irrespective of what classifier you dream up. Unless your classifier is not independent from the base rate - that is, a classifier that gets more lenient if there's more fraud in the group. That seems undesirable when considering fairness as a goal.

[tripletao]

> Unless your classifier is not independent from the base rate - that is, a classifier that gets more lenient if there's more fraud in the group.

If I understand correctly, that was the purpose of their "reweighting". To remove "bias" from their new model, they looked at the aggregate false positive rates by nationality, then applied explicit penalty to people with Western nationality to bring that FPR equal to non-Western.

I'd certainly agree that's undesirable, and my point was also to highlight that (since the article was reasonably clear on the type of fairness they achieved, but much less clear on the fairness they sacrificed).