[elevensies]

I would phrase it this way:

The law says racism is illegal in certain situations, and society says racism is undesirable in most situations.

The law and society aren't claiming that racism is statistically non-optimal -- in fact, there are lots of things more optimal than status quo that many people would find totally horrifying.

If are widely replacing human systems with AI systems, I think this is a legitimate concern.

I really depart from the article in two areas:

1. The AI will inherit the biases of its creators. This is possible but far from guaranteed. And relatedly, inclusivity of the development team guarantees nothing regarding the goals of the system.

2. Criticising the people who are warning of the problem and trying to do something about it. This is related to the AI control problem. There is no switch that can be flipped that will prevent AI systems from Bad Ideas. It's not that we just aren't flipping it to preserve our chokehold on captialism. Implementing morality in AI systems is a genuinely monumental problem. And the people who are doing something about it are behaving very altruistically.

[bonobo3000]

Agreed! If we did nothing to correct inequalities in society, the biggest and strongest would rule over all - so yes we must have our own values and stick to them.

And with AI, we must make these values explicit, which is very difficult to do - i agree this is a very important problem to solve, and the people doing it should absolutely be rewarded.

Honestly, reading the article again after your summary i found it very reasonable :p I think the headline just ticked me off.

No one is stopping other people from getting in on the debate, they absolutely should (and I'm sure there are roadblocks in their way, and people who really are racist). It just feels wrong to implicitly all the "bad white people" for that. Blame those who cause the problem. Otherwise we are back to stereotyping.

[jimmywanger]

Certain terms are ill defined in your statement.

"The law...". Whose law? "Society says...". Which society? "Implementing morality". Whose morality?

Also, why wouldn't you want things to be optimal, depending what what they're optimizing for? I thought optimizing was the exact point of machine learning and AI.

[elevensies]

The laws I had in mind when writing are US protected classes: https://en.wikipedia.org/wiki/Protected_class

By society I was thinking of Western society. Wikipedia lists equality as value in 2nd paragraph here: https://en.wikipedia.org/wiki/Western_culture

By morality I was thinking along the lines of socially accepted behaviour in the contemporary United States, and mainstream Christian morality which isn't the official state religion, but in my opinion the basics of it are taken as a given in politics, government, media, and academia.

Overall, I'm referring to mainstream anglo law, society, and morality.

> why wouldn't you want things to be optimal, depending what what they're optimizing for?

The reason is that when you express your goals, you don't fully understand what the consequences will be. There may be consequences that are totally repugnant to you. The child-story level version of this is where you get a genie which gives you your wishes in a horrible way: i.e. I wish to be the richest man in the world, and so the genie kills everyone else. I want a nice big house like my parents, and so the genie kills your parents and you inherit. Et cetera.

Telling the computer to do what you want is notoriously difficult for simple imperative programming, to the point where many people think a large fraction of the population just isn't up to it intellectually, and if you need proof of this you can search for fizz-buzz interview stories. Setting up goals or incentives for a system that behaves in a way the you can barely understand is even more difficult.