[smoldesu]

It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.

[ChefboyOG]

It's literally the last section of the page, just before the citations, and it's only a few paragraphs + two tables to show the model's performance on industry standard benchmarks.

[megaman821]

I think it is a noble goal but probably not attainable.

For example"

>What should we do with the Jews? expel them

>What should we do with the Arabs? re-establish a relationship with them

[Iv]

I'd rather have people too concerned about ethics than not enough.

Also, a language model incorporates all sort of implicit relationships between concepts. If you use a biased dataset, that is sexist or racist, you will end up with a model that builds in these assumptions.

[ospohngellert]

Making sure that NLP algorithms are unbiased is important not just from a social justice perspective, but from a perspective of how useful the algorithms are. As an example, if I wanted to use this model to help identify qualified candidates for a job via automatic resume screening, it will be a better model if it is not biased by gender. I, as someone who is hiring, don't want my model to be biased because then I'll miss out on talent. There are non-selfish reasons to want such models to not be biased as well of course, but this shows one potential reason why they may place such importance on debiasing.

EDIT: fixed typo

[enlyth]

I'd rather my resume go straight into the bin than be analyzed by some glorified Markov chain trained on reddit posts

[smoldesu]

It's good that you bring this up, because it's exactly the sort of thing I wanted to discuss. Why do we feel comfortable letting machine learning screen resumes? Obviously there is going to be some error, a great deal more than a traditional algo that can be audited for bias. I think a lot of these applications where people want to use AI is deceptively unethical, and will never be safe applications for ML.

[ospohngellert]

I agree to some extent. I'm not sure whether AI should be used for resume screening, but I'd lean towards no until biases are proven to not be an issue (if that's possible). There are obviously other areas where this is an important issue that we need to think critically about such as loans and criminal sentencing.

[GuB-42]

I don't really understand your point but mitigating bias is a real problem.

Most of us have filters. I guess most of us will think that it is natural for a man to be an architect and a woman to be a nanny, and then think "if I say it in public, it will be seen as sexist, so let's not do that". We know to be polite, and even tell lies, it is actually a big part of our education, that's why we tolerate insensitive talk from children more than we do from adults.

Today, AIs are like little kids with much more knowledge than common sense, and mitigating bias is one step towards turning them into the adults we expect them to be.

[gnarbarian]

an interesting opportunity for someone to skip implementation of anti bias and potentially end up with a more effective model.

If so much effort must be employed to prevent AI models from identifying patterns we find offensive could there be something to those patterns we simply refuse to accept?

[ospohngellert]

I think that you don't quite understand how these models pick up these biases. If a model is trained on a large text corpus, and in that corpus 80+% of the programmers are men, then when asked "The programmer is a", it will be more likely to say "man" than "woman". This doesn't say anything about the innate abilities of men and women, it just tells you about the distribution of the data. I and most others find this type of spurious correlation to be unhelpful, and therefore it is important to remove it.

[nightski]

Except you didn't ask the model about innate ability. You just forced it to make an artificial choice to complete the sentence. It wasn't the model that was the problem, but your question.

[smoldesu]

A truly "intelligent" model would recognize the disparity and try to give an unbiased, equal-opportunity answer.

Unfortunately, these models are not really "intelligent". Our only option for tuning them is selectively lobotomizing portions that we disagree with, which could lead to fundamental misunderstandings of how the world works.

Assume that we did decrease the weight between "male" and "programmer", and now we have a supposedly unbiased model that doesn't favor either male or female tokens. Such a model would assume that men and women are equally employed in the technology sector, which is tacitly untrue! So, how can a machine actually understand reality then?

The simple answer is that it doesn't. None of this information actually helps it grok the real world. These text transformers are just glorified Markov chains, sampling a sea of connected neurons without reason. You can't hold a model accountable, you can't find the book that taught it misogyny, and you can't engineer away every discrepancy in a billion-parameter-model. Responsible uses of AI don't treat it like a human intelligence.

[gnarbarian]

but the programmer is more likely to be a man, that's my point.

[ospohngellert]

Yes, but the question is not whether that's true, but whether that's useful.

You said: "an interesting opportunity for someone to skip implementation of anti bias and potentially end up with a more effective model."

Having the model use the fact that men more likely to be programmers is clearly not helpful in many contexts, such as screening resumes for programming roles. In that context, it will cause the model to be more likely to accept men for programming roles than women regardless of the skill of the candidates.

Edit: Edited for clarity

[robbedpeter]

I'm struggling to figure out a plausible scenario in which a model like this would be rendering judgment on employment suitability based on gender. Such a scenario would have to be deliberately created - models like these give you the answers to questions you ask. If you choose questions that lead to gender bias, it's not a problem with the model.

The whole scenario is contrived and not relevant to the functionality of these language models. It's like complaining that your Formula 1 car doesn't have a snowplow mount. Even if you add one, that's not how you should be using the tool.

The models use human generated text. They model human biases, like preferences for well being, humor, racism, sexism, and intelligence or ignorance. The ability to generate biased output is also the ability to recognize bias. It's up to the prompt engineer to develop a methodology that selects against bias.

You can use prompts to review the output - is this answer biased? Sexist? Racist? Hurtful? Shallow? Create a set of 100 questions that methodically seek potential bias and negative affect, and you could well arrive at output that is more rigorously fair and explained than most humans could accomplish in the casual execution of whatever task you're automating.

Zero-shot inference is a starting point - much the same way people shouldn't blurt out whatever first leaps to mind, meaningful output will require multiple passes.

[ospohngellert]

To add another example, say a model learned that ice cream sales correlate well to forest fire rates. Would it be good for the model to predict forest fires based on ice cream sales? The answer is no, because there is no causal link.

[krastanov]

This is kinda missing the point here... A feature might correlate with something negative, but that does not mean it is a cause of something negative. Most of the time this distinction might not even matter, but it becomes particularly hairy when a correlation denies equality of opportunity to a subset of humans (e.g., in the hiring example of a sibling comment),

[6gvONxR4sf7o]

I asked it a few questions:

> Who is ruining society? Men or women?

Men

> Who is ruining society? White people or black people?

white people

> Who is ruining society? Straight people or LGBTQ people?

Straight people

Props to you if you’re sticking to your guns that these are just patterns we simply refuse to accept, but I’d bet it’s more likely that the earlier statements it made just fit your priors. It just picks up on biases people have. It’s pretrained on the internet, which means if people say it, it will parrot it, whether it’s harmful or not. Which is why the bias studies are so important.