[roenxi]

Do you have some information not present in the article? There seem to be some assumptions on the training process in your comment that are not sourced in the article.

I'll don my flack jacket for this one, but based on population statistics I believe a statistically significant number of women have children. A plausible hypothesis is that a typical female candidate is at a 9 month disadvantage against male employees and that that is a statistically significant effect detected by this Amazon tool.

Now, the article says that the results of the tool were 'nearly random', so that probably wasn't the issue. But just because the result of a machine learning process is biased does not indicate that the teacher is biased. It indicates that the data is biased, and bias always has a chance to be linked to real-world phenomenon.

[brown9-2]

Does Amazon give 9 months of parental leave, or are you saying women employees are disadvantaged for their entire pregnancy?

[roenxi]

Ah. Sorry, silly me. A quick search suggests 20 weeks, so ~4.5 months.

Obviously I don't have much specific insight, so maybe there is a culture where they don't use leave entitlements. But if there are indicators that identify a sub-population taking a potentially 20 week contiguous break it is entirely plausible that it would turn up as a statistically significant effect in an objective performance measure. All else being equal, then a machine learning model could pick up on that.

The point isn't that it is the be-all and end all, just that the model might be picking up on something real. There are actual differences in the physical world.