[mlyle]

> that there may be some implicit language bias which relates to race in there.

I think there is some stuff in the middle. I think disadvantaged groups deal with more of these bugs by being underrepresented in the teams that design this stuff.

Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of.

Especially once we keep getting adverse outcomes and don’t manage to prioritize fixing it.

[kelseyfrog]

> Are soap dispensers that don’t give soap to people with darker skin racially biased? Kind of.

I'm struggling to extract the nuance that I feel is embedded here. Is it an ontological issue, a quantitative, or a qualitative dimension that puts it in the realm of kind-of?

[malfist]

Vision based tech developed in silicon valley notoriously doesn't often include a diverse training sample. There are have been more than a few cases were technology has only been tested on light skinned people and released to the world and didn't work for anyone with more pigment in their skin.

Soap dispensers that detect hands are one example, highlighted by GP, I believe Microsoft's Kinect also had issues on release detecting non-white people.

[Cheer2171]

Yeah, why "kind of" and not "absolutely"?

The black box systematically changes the output based on the presence of an input signal of skin tone, which is strongly correlated with race.

Same with photographic negative film, where engineers made a trade-off with the chemicals that made contrast much better for lighter-skinned faces, which made darker-skinned faces much worse. In group portraits, darker-skinned people would appear as just teeth and eyes. Although in true American style, it was chocolate and furniture companies (and their advertisers) who pushed Kodak to develop better technology for capturing dark brown colors.

https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b...

[mlyle]

I think it's "kind of" because part of the problem is intrinsic; if you're trying to detect reflections, and you're not even reliable with light skin 100% of the time, it says that there's some inherent difficulty here.

You realize it's biased when you think about it and realize the product could never have successfully shipped if it offered white people the poor reliability that it has for darker skin.

[tacitusarc]

There is a large amount of academic literature claiming racial groups are purely ethnocultural, and in that light it is ridiculous to claim a soap dispenser is racially biased.

[wizzwizz4]

The US has its own special meaning of "race", "racial group" and similar – but, since societal groups are what society thinks they are, this means that "racial groups" in the US are (in large part) defined by things like skin colouration. In that context, claiming that the soap dispenser is "racially biased" is a perfectly good description.

If you perform a root-cause analysis (a soap dispenser doesn't exist a vacuum), you will discover "racial bias" in the structures and institutions that designed and created the soap dispenser. Metonymically, therefore, it's even more accurate to claim that the soap dispenser is "racially biased".

(Author's note: I continue to reject the US's conception of "race". Please do not understand this comment as legitimising it in any way.)

[mlyle]

Say you're making a soap dispenser. It doesn't work for some people. You don't really care or notice because they're members of racial groups that you actively dislike or maybe just don't really care about.

Couldn't the soap dispenser still be an instrument of racial bias, even if this racial group is an ethnocultural distinction?