[nzeribe]

I find it baffling and dispiriting that you, as a Black person, believe this is more likely to be an "error" than an act of deliberate racism. It's getting silly, now. A white racist programmer has done this on purpose because he finds it hilarious. The chances of a computer equating monkey and man, given the correct datasets, are close to nil. There's a blindness - a wilful ignorance - to look away and show good faith. Why is that?

[jpambrun]

As someone who worked on ml based classification I can totally see a man being misclassified as a monkey on occasion. If you have man and monkey in the output vector, it will happen on from time to time.. If you find a way to achieve 100% accuracy, go claim your Nobel prize..

[nzeribe]

But that's not what happened, is it? The algorithm hasn't miscategorised men, it has miscategorised Black men. I'd argue the AI has done its job, with a high degree of accuracy. It found what the programmers were looking for when they wrote the code. It was meant to be an insult, and was highly effective. Lack of outrage on HN is as expected; the authors are quite possibly in the comments holding their sides with laughter.

[jpambrun]

Those are baseless accusations based on poor understanding of ML.

This behaviour is counterproductive. It is not susceptible to sway me towards your point of view, but instead discredit you and those holding similar opinions.

[nzeribe]

I am not here to "sway" you. Your hand-wavey dismissal is absurd, given the history of race in Silicon Valley. William Shockley was an out-and-out white supremacist. He goes on to form Shockley Semiconductor which begets Fairchild which becomes Intel. He set the culture. It is patently obvious that this outcome is likely written in code, and for you to dismiss this probability is absured. You have not discredited my position in any way, nor even attempted to challenge it. How on earth would you know if my understanding of ML is "poor" or otherwise? The joke in the code was to associate Black Men with monkeys. It is a trope that goes back hundreds of years. The joke is: "Look, even computers think Black men are primates" and that this position is objective, logical.

[jessaustin]

Some people want to police AI behavior. Others want to police people's opinions of AI behavior.

[jrm4]

Actually I now must strongly depart from your argument of intentionality here, and reiterate something I said before. I was about to respond to the person below, but they're more-or-less right. I don't know how intentional any of the programmers were here, so I'm not going to speculate on that.

But I will absolutely insist that this racist outcome is possible -- even likely -- absent any bad intentions from the programmer(s) who did it.

ALSO: I'm 99% sure of the following:

A black team of programmers working on "distinguishing humans from monkeys" would never let the mistake of "black and not white people identified as monkeys" out the door.

That's the point here. I'm not saying it wouldn't have happened somewhere in the process, I'm saying that you or I (if we don't work on the inside) would NEVER have seen it because it would have been noticed and fixed before then.

[nzeribe]

Why wouldn't you assume intentionality as more likely than not given the disturbing history of race in Silicon Valley? The position you hold out of the gate is unlikely and your assumptions of good-faith are misplaced. William Shockley, who invented the transistor and essentially set up the Valley and it's culture, was an outright white supremacist. You see his legacy in the hiring percentages. The probability is that this was done on purpose. Your position is dangerous, in all honesty, because it sets up the idea that there is a logical similarity between Black Men and monkeys. "Look! Even a computer thinks so!" - and that we must modify outcomes so spare Black people's feelings. Note that the alleged error wasn't in distinguishing humans from monkeys as you state in error, but Black Men specifically.

[jrm4]

Primarily because my way is more forward looking and gets better results in the future, which is more important than punishing the past.

As I said before, I'm not here to smoke out old (or even current) racists individually. I don't think it's a valuable practice to "witch hunt" (even when the "witches" in this case are absolutely real and do exist.)

Because what that ends up doing is: every e.g. white person who's never said the n-word, or who has black people in their family, or has one black friend etc etc etc now subconsciously but completely lets themselves off the hook in any way. They get to think of themselves as superior because we've now defined racism as essentially a binary.

The above situation is mentally easier for many people (perhaps you as well) to deal with, rather than considering how deep this all goes.

Look, one wild thing you realize as a black person is that nearly everybody everywhere is to some extent surrounded by racism is that most everybody has some of it subconsciously internalized, and you don't fix it until you think about it directly, in yourself and others. I'm not a huge fan of "oh everybody's a little racist" because whoever says it is usually doing something dumb like excusing behavior -- but it's FAR closer to the truth than "if we smoke out the hidden but self-consciously racist people all will be fixed."

(so I suppose I'm saying -- yes, pay attention to what you are suggesting we pay attention to -- but also understand that it is almost CERTAINLY nowhere near sufficient to fix the problem.)

[jrm4]

Yup. Though I would refine it a bit -- it's the "rush to excuse" that's the most dangerous. I don't much worry about the deliberate racist programmer as much as I worry about subconscious deliberate-or-not biases creeping in.

This perfectly analogizes to most everywhere else; angry open n-word saying racists are usually powerless losers.

More problems come from the larger combination of the rest of the "racism" spectrum, whether apathetic, or racist-but-quiet, or harbors latent biases that they may not know about etc.

(And here I do feel like I have to say, the answer isn't "SMOKE THEM OUT AND EXPOSE THEM" on the personal level, it's just taking the utmost care in the work you do)

[pure_simplicity]

In this case, it's not the rush to excuse that is dangerous, it's the rush to accuse that is dangerous. Innocent until proven guilty is a fundamental principle of justice. You have not demonstrated beyond reasonable doubt that this is in fact the doing of a racist, so anyone coming up with plausible alternative explanations is only doing so to prevent the discussion from devolving into a witch hunt where the guilt of the accused is simply assumed and our biases and perceptions stand in for evidence. Repeat the witch hunt scenario often enough and eventually you end up with a boy who cried wolf scenario and now you've lost your credibility even in cases where you do have evidence to back up your claims.

When people are careless in making accusations that's incredibly damaging to society. You as a black person should know how divisive and damaging false allegations can be, since that was a tool racists used to instill racial hostility in society. Let's not repeat those mistakes. I'm not saying racism doesn't exist or that we shouldn't do anything about it, but knowing in your heart of hearts that this must have been an act of deliberate racism is simply not enough. We need a smoking gun, or else it's best to withhold judgment.

The fact that no one thinks the mislabeling by the AI is acceptable should be enough. That shows you that no one would defend a person who deliberately designed this. The 'rush to excuse' is not because society doesn't want to condemn racism, but because most of society still cares about fairness and not making false accusations.

[jrm4]

If you follow who is saying what, I don't think you and I fundamentally disagree - or at least are not THAT far apart on this - I (the black person here who is still saying yep, my black opinion is important) have also been strongly suggesting that we not automatically attribute these problems to intentional racism.

So to clarify -- the "rush to excuse" that I'm saying is dangerous is really "the rush to insist that there is definitely no bias in play," NOT the "the rush to accuse individuals of being racist" -- and I suppose the fundamental problem I see going on here is that people can't seem to distinguish these two things, even though they are very different.

I'm going to keep saying this: the biggest problem is not smoking out hardcore racists who openly hate, the biggest problem is getting much of the non-black tech populace to even begin this conversation without the hypersensitivity kicking in (fully acknowledging that, while I'm not a fan of the terms "SJW" and "woke" and such, there also absolutely exists a naive liberal left that presently makes this conversation harder because they lean too hard into their particular direction)

[jrm4]

And now I'm very curious as to the reasoning behind the downvotes. Again, I'm fine with you disagreeing with me, but I must insist that discussion is more valuable than blind downvotes when you do have the rare occasion of an actual real life black person here giving their opinion.