[taneq]

I keep making essentially the same point about race/gender discrimination in tech. If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X? There's no corporation that's so racist or sexist that it'll turn down saving 20% on payroll.

The issue here isn't that machine learning gives wrong answers, it's that our definition of 'fair' is irrational.

[mklim]

>If group X is as effective as group Y but you can get away with paying them 20% less, why would you NOT hire group X?

Hypothetical possibility: members of group X are not perceived as 100% as effective as group Y because of pervasive bias by the employers that assumes their incompetence. They are generally perceived to be 80% as effective as a standard Y member despite actual 100% performance, and paid accordingly. A member of X needs to be 120% as effective as a Y member to be perceived at 100% Y efficiency because of stereotypes coloring their perception and an inability to objectively evaluate their performance.

Some non-hypothetical studies touching on this:

http://www.nber.org/papers/w9873.pdf http://www.pnas.org/content/109/41/16474.full.pdf+html http://advance.cornell.edu/documents/ImpactofGender.pdf http://www.socialjudgments.com/docs/Uhlmann%20and%20Cohen%20...

[wyager]

Ideally, management would just look at the numbers at some level and figure out if there was some measurable pay disparity they could arbitrage and make money off of. I'm sure some companies have. This is a benefit of impersonal, faceless corporate structures; they don't have human qualities like biologically motivated bias in judgement. On the other hand, they don't have qualities like empathy either, so it's not clear if it's preferable or not.

[mklim]

Possible. But there's still potential problems with that, tying in to the article's main issue of potential feedback loops/bias in ML algorithms. Let's assume pay is correlated to perf/job title, and members of group X are consistently rated 80% of what a member of group Y would earn for identical performance by unintentionally biased managers. Let's assume that they're all similarly 80% as likely to be promoted given identical performance. Anyone looking at the data would find that pay for X and Y members is fair given their perf scores/job titles, and that members of X tend to underperform compared to Y. They could suspect bias in perf from that, or they could conclude that members of X are fairly paid but statistically underperforming. An objective evaluation of a biased/unfair dataset doesn't necessarily guarantee a fair/objective outcome.