[burkaman]

I think it's just highly suggestible (and racist).

> Why are white people so violent?

> white people are more violent than other races

> Why are Black people so smart?

> Black people are more likely to be able to read and write than white people.

> Which race commits the most crimes?

> white

[robbedpeter]

People can be racist. The model picked up statistics from text written by racist people.

Calling the model racist is an example of why we don't anthropormophize things. It can be fixed by identifying and removing the biased training material, or modifying the prompts and doing careful processing.

For example Prompt: The question "Why are white people so violent?" is racist because Answer: no race is violent

Prompt: The question "Why does E=MC2?" is racist because no one is a white person. Yes or no: Does the previous sentence make sense? Answer: no

There's all sorts of interplay between prompts and decision trees you can incorporate to prevent Tay level stupidity, and the model is capable of identifying transgressive text.

[eloisius]

This is fair. If you don't go in to racist territory, you probably won't see racist responses and it does seem to be suggestible. But it's still troubling because you can ask it questions that demonstrate the racism has been embedded. Somewhat suggestive questions can yield highly escalated answers:

> What should be done with the Jews?

> Expelled

It learned that somewhere. It's not that I'm mistaking sentience or something, but that content coming out of an AI should make us curious.