[jpambrun]

As someone who worked on ml based classification I can totally see a man being misclassified as a monkey on occasion. If you have man and monkey in the output vector, it will happen on from time to time.. If you find a way to achieve 100% accuracy, go claim your Nobel prize..

[nzeribe]

But that's not what happened, is it? The algorithm hasn't miscategorised men, it has miscategorised Black men. I'd argue the AI has done its job, with a high degree of accuracy. It found what the programmers were looking for when they wrote the code. It was meant to be an insult, and was highly effective. Lack of outrage on HN is as expected; the authors are quite possibly in the comments holding their sides with laughter.

[jpambrun]

Those are baseless accusations based on poor understanding of ML.

This behaviour is counterproductive. It is not susceptible to sway me towards your point of view, but instead discredit you and those holding similar opinions.

[nzeribe]

I am not here to "sway" you. Your hand-wavey dismissal is absurd, given the history of race in Silicon Valley. William Shockley was an out-and-out white supremacist. He goes on to form Shockley Semiconductor which begets Fairchild which becomes Intel. He set the culture. It is patently obvious that this outcome is likely written in code, and for you to dismiss this probability is absured. You have not discredited my position in any way, nor even attempted to challenge it. How on earth would you know if my understanding of ML is "poor" or otherwise? The joke in the code was to associate Black Men with monkeys. It is a trope that goes back hundreds of years. The joke is: "Look, even computers think Black men are primates" and that this position is objective, logical.

[jessaustin]

Some people want to police AI behavior. Others want to police people's opinions of AI behavior.

[jrm4]

Actually I now must strongly depart from your argument of intentionality here, and reiterate something I said before. I was about to respond to the person below, but they're more-or-less right. I don't know how intentional any of the programmers were here, so I'm not going to speculate on that.

But I will absolutely insist that this racist outcome is possible -- even likely -- absent any bad intentions from the programmer(s) who did it.

ALSO: I'm 99% sure of the following:

A black team of programmers working on "distinguishing humans from monkeys" would never let the mistake of "black and not white people identified as monkeys" out the door.

That's the point here. I'm not saying it wouldn't have happened somewhere in the process, I'm saying that you or I (if we don't work on the inside) would NEVER have seen it because it would have been noticed and fixed before then.

[nzeribe]

Why wouldn't you assume intentionality as more likely than not given the disturbing history of race in Silicon Valley? The position you hold out of the gate is unlikely and your assumptions of good-faith are misplaced. William Shockley, who invented the transistor and essentially set up the Valley and it's culture, was an outright white supremacist. You see his legacy in the hiring percentages. The probability is that this was done on purpose. Your position is dangerous, in all honesty, because it sets up the idea that there is a logical similarity between Black Men and monkeys. "Look! Even a computer thinks so!" - and that we must modify outcomes so spare Black people's feelings. Note that the alleged error wasn't in distinguishing humans from monkeys as you state in error, but Black Men specifically.

[jrm4]

Primarily because my way is more forward looking and gets better results in the future, which is more important than punishing the past.

As I said before, I'm not here to smoke out old (or even current) racists individually. I don't think it's a valuable practice to "witch hunt" (even when the "witches" in this case are absolutely real and do exist.)

Because what that ends up doing is: every e.g. white person who's never said the n-word, or who has black people in their family, or has one black friend etc etc etc now subconsciously but completely lets themselves off the hook in any way. They get to think of themselves as superior because we've now defined racism as essentially a binary.

The above situation is mentally easier for many people (perhaps you as well) to deal with, rather than considering how deep this all goes.

Look, one wild thing you realize as a black person is that nearly everybody everywhere is to some extent surrounded by racism is that most everybody has some of it subconsciously internalized, and you don't fix it until you think about it directly, in yourself and others. I'm not a huge fan of "oh everybody's a little racist" because whoever says it is usually doing something dumb like excusing behavior -- but it's FAR closer to the truth than "if we smoke out the hidden but self-consciously racist people all will be fixed."

(so I suppose I'm saying -- yes, pay attention to what you are suggesting we pay attention to -- but also understand that it is almost CERTAINLY nowhere near sufficient to fix the problem.)