[yummyfajitas]

Do you have any evidence that this effect results in machines making systematically wrong inferences?

Near as I can tell, your paper shows that these "biases" result in significantly more accurate predictions. For example, Fig 1 shows that a machine trained on human language can accurately predict the % female of many professions. Fig 2 shows the machine can accurately predict the gender of humans.

Normally I'd expect a "bias" to result in wrong predictions - but in this case (due to an unusual redefinition of "bias") the exact opposite seems to occur.

(Drawing on your analogy with stereotypes, it's probably also worth linking to a pointer on stereotype accuracy: http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-... http://spsp.org/blog/stereotype-accuracy-response )

[rmxt]

Accuracy might mean "positively" right, as your post suggests, but that doesn't necessarily mean "normatively" right.

From what I understand, the fear surrounding embedding human stereotypes into ML systems is that the stereotypes will get reinforced. In some way or form, there will be less equality of opportunity in the future than exists today, because machines will make decisions that humans are currently making. Societal norms evolve over time, yet code can become locked in place.

Is your takeaway from this paper that we, as the creators of intelligent machines, should allow them to continue to making "positively" right assumptions simply because that's the way we, as humans, have always done them? Is "positively" right, in your opinion, in all cases equivalent to "normatively" right?

[andreasvc]

I think your questions would be answered by reading the article. Particularly:

"In AI and machine learning, bias refers generally to prior information, a necessary prerequisite for intelligent action (4). Yet bias can be problematic where such information is derived from aspects of human culture known to lead to harmful behavior. Here, we will call such biases “stereotyped” and actions taken on their basis “prejudiced.”"

This definition is not unusual. This is about inferences that are wrong in the sense of prejudiced, not necessarily inaccurate.

[yummyfajitas]

The usual definition of bias in ML papers is E[theta_estimator - theta]. That is explicitly a systematically wrong prediction.

In any case, the paper suggests that this "bias" or "prejudice" is better described as "truths I don't like". I'm asking if the author knows of any cases where they are actually not truthful. The paper does not suggest any, but maybe there are some?

[andreasvc]

Again, per the article "bias refers generally to prior information, a necessary prerequisite for intelligent action (4)." This includes a citation to a well-known ML text. This seems broader than the statistical definition you cite.

Think for example of an inductive bias. If I see a couple of white swans, I may conclude that all swans are white, and we all know this is wrong. Similarly, I may conclude the sun rises everyday, and for all practical purposes this is correct. This kind of bias is neither wrong nor right, but, in the words of the article "a necessary prerequisite for intelligent action", because no induction/generalization would be possible without it.

There are undoubtedly examples where the prejudiced kind of biases lead to both truthful and untruthful predictions, but that seems beside the point, which is to design a system with the biases you want, and without the ones you don't.