[theoh]

Consider the possibility that the (pre-AI system) probability of success for a female applicant is the same as the probability of success of a male applicant. You could make a "per capita" quota as a kind of goal. That's not a problem, but how would you make sure the quota was met?

The typical AI system doesn't work on the basis of selecting candidates entirely at random, pro rata, in order to meet a quota. It works on the basis of criteria for success. One thing it might learn (unfortunately) is that most posts at the company are filled by men.

[cygaril]

From a machine learning point of view, one can just add the constraint that the probability of being in the "yes" bucket is that same for both male and female candidates. Doing this will give a worse fit than an unconstrained optimization, but it is fairer.

More sophisticated approaches are possible.

[theoh]

There's no "just" to any aspect of this topic. I think what you are talking about is what is sometimes called "classification parity", and there are problems with it, and with everything else we've come up with to combat bias.

https://arxiv.org/abs/1808.00023