[jsinai]

> But she didn’t show how this is harming anyone in a concrete way.

To roll with the example “man” ~ “doctor”, “woman” ~ “nurse”, the harm is having a giant and widely used search engine reinforce baseless gender biases, ie that there is no underlying reason why women should be nurses and men doctors. What is the harm you may ask? The harm may be subtle, eg being surprised when you find out your next doctor is a woman or your next is a man. It could suppress career choices and aspirations, and it could even be financial, eg reinforcing systemic pay gaps.

[thu2111]

That particular "bias" isn't actually a bias, it's an accurate learning about the distribution of genders between jobs in the real world. Very few professions have an exactly 50:50 balance of men and women. Most are tilted towards one gender or the other. The purpose of a correct search engine is not to reduce my "surprise" at arbitrary events but rather to give me the information I'm looking for, which will more often than not be questions about the real world - not the fever dream of some hard-left activist.

[s__s]

Indeed. To intentionally skew the data such that, for example, men are over represented as nurses, is in fact introducing bias to the data based on prejudice.

You’ve essentially created a fictional data set because it’s biased due to the underlying prejudice (preconceived opinion that is not based on reason or actual experience) that men ought to be nursing more, despite that not being reality.

We’re in a strange situation where we have large concerted efforts by activists to inject fiction in to our facts (whatever the medium) with the aim of distorting perceptions in such a way as to some how correct what they perceive to be injustice in the real world.

[visarga]

This kind of hypothetical effects should be documented in concrete cases by a good ethical scientist instead of just described from imagination. For example, when someone searches for a doctor, let's say a male comes up first - who stops at the first Google result? They would probably need to go deep and read about the doctor's experience and find patient reviews.

[jsinai]

Part of the issue is that the harms of gender bias (and other types of bias) should not need to be made explicit, but part of the research canon. Should a security researcher outline the harms of an attacker obtaining user credentials, or is our imagination sufficient because the harms are well known to us? And if you were looking for more in depth studies, then there is a ton of published research, maybe not all of it on arxiv or in machine learning journals.

[visarga]

At some point imagination has to make touch with reality otherwise it can become unhinged. Yes, security researchers can enumerate concrete cases where "the harms of an attacker obtaining user credentials" caused damage.