[ghosty141]

It produces some "interesting" results. I put in "robber" expecting that one emoji everybody thinks exists but doesnt, and I got a black guy with a white beanie/hat.

Apart from that it produces insanely convincing results. I believe people would not bat an eye if they'd get added in an update to iOS!

[sushisource]

Heh, I put in "grapefruit" and it gave me just an absolute mess of like 100 grapefruit slices and then added a bunch of smoke over the top when it "removed the background".

Then I tried "day gecko" and the output absolutely rocked. So I guess my first try was unlucky.

[SamBam]

It loves giving me 100 of someone. I'm wondering if there are sticker pack-like images in it's training set.

And then, yes, the background removal takes away half of these and leaves smoke.

[Charlieholtz]

Yeah, that’s a good point. We’re currently training a new version that reduces this bias. It’s really important to us that we make this better.

[ChatGTP]

Drug dealer, car thief, car jacker, drive-by-shooter, murderer...all of those things and many other crime related terms will give you a black person.

Quite a shit thing to be happening.

[fnordpiglet]

AI models and their biases are reflections of the biases of society. I don’t like them, but it’s hard to argue with the fact it’s just math producing the semantic best likelihood based on publicly sourced training data.

[chefandy]

See that's the thing about content creation— you entirely lose the "it's just a tool and tools are inherently neutral" argument. Nobody's claiming that the implementor created deliberately racist content, but you can't merely disavow responsibility for the content you generate because the input and algorithms you chose were the root of the problem. Like it or not, what's included in the source data for these models is an editorial choice, and which model you use and how you use it are also editorial choices.

[fnordpiglet]

Actually I think “it’s a reflection of society’s bias” is a totally reasonable statement to make if you’re product is a reflection of societies generated content.

Rather than impugning the model makers for not curating societies content to erase its biases, to my mind it demonstrates what’s broken with society, and should be used as an indictment of how we encode our society in our media - the fact that if you asked an oracle a question it produces racist output is more an indictment of society itself because the media of our society IS racist. Sweeping it under the rug serves no one, IMO.

Instead the story is “AI models are racist,” which misses the real problem. The real problem is that when a human in our society wants to portray a robber, they use a black man. That should be the story, and pitching it as a flaw with AI models is like criticizing the color of the paint when the foundation is cracked.

[lawlessone]

How does perpetuation of stereotypes fix society?

[reidjs]

It's not perpetuating stereotypes, it's showing that these stereotypes exist in society. Similar to how a comedian might point out the absurdity of racism by writing and delivering a joke about race. Or how a child might ask why people of certain skin tones tend to have different hair styles as well. Neither the child nor the comedian are definitely racist, but they are making observations that may not be considered politically correct.

[pizzafeelsright]

What is wrong with stereotypes?

Snakes are, at times, quite dangerous. Granted, not all snakes, but in terms of being risk-averse, and unaware of which snakes are quite possibly deadly, am I not, at least in my current ignorant state, best served to avoid all snakes?

Feel free to substitute "snakes" with any existing stereotype. I personally am a fan of "people who eat pizza with pineapple".

[lawlessone]

>AI models and their biases are reflections of the biases of society.

It's not one way though it perpetuates the bias. I'm not blaming the author of this as its Stability AI that need to clean their data.

[Charlieholtz]

Yeah I’ve also noticed an unintended bias that we’re not proud of.

We’re on it – we’re reviewing the model and fixing the issue ASAP. Thanks for pointing it out and for your patience!

[staringback]

Just take a step back and think about what you've said and the meaning of the word "bias"

[lawlessone]

:( Everyone really needs to clean their data

[pizzafeelsright]

Who will clean the cleaners? Are they without bias and without blemish?