[sanxiyn]

>> there is no magic algorithm that bypasses this fact.

> Maybe there is

No, there isn't. We actually have a mathematical proof (which is quite simple) why this is impossible.

Specifically, following conditions can't be true at the same time: 1. groups differ in base rate 2. prediction isn't perfect 3. decision is correct at the same rate for groups 4. decision is correct at the same rate for groups, restricted to positive/negative class.

1 is a brute fact. Your toy example insinuates at 2. 3 is called calibration and what is usually optimized by machine learning. When people say algorithm is unfair, it usually means 4.

https://arxiv.org/abs/1609.05807