[timmg]

> There has been plenty of research that shows LLMs encode social biases.

At the risk of stepping into a hornets nest: is that different than "knowledge"?

Or maybe, what would it mean if an LLM had no social biases? (Would we ever agree that was the case?)

[tptacek]

Yes, it would be extremely bad if the statistical weight of the total corpus of training data caused a system using an LLM to make decisions about extending credit to offer worse terms (say) to women.

[timmg]

> sing an LLM to make decisions about extending credit to offer worse terms (say) to women.

In general, or if it isn't the correct answer?

Like: young men pay more for car insurance than young women (today). This is based on statistical models. Should they be outlawed? I think that is a very interesting question (but they aren't, today).

If the LLM was in charge, would it be wrong for it to charge young men more? Should we train that "bias" out? Or should we only train out biases that are wrong? And would that be different than how we train them today?

I don't know the answer. But I think it is less obvious than some people seem to think.

[em-bee]

young men pay more for car insurance than young women (today). This is based on statistical models. Should they be outlawed?

EU has outlawed them. their argument is that differentiation is only valid if the difference is the actual cause and not merely statistical correlation.

[timmg]

Ironically, in the US it is ok to charge men more for car insurance, since they cost more in aggregate. It is illegal to charge women more for health insurance even though they cost more in aggregate.

[em-bee]

given the economic realities of income between men and women, i think that makes sense.

[tptacek]

It would obviously be very bad if those decisions were being made based on the statistical weight of the training corpus of a general large language model.

[tsss]

That just shows how biased you yourself are. Every human is. It is FAR more likely that the algorithm would give better credit terms to women and worse terms to men, as it is already the case with insurance. Yet you assume the opposite because of your personal biases.

At least LLMs offer a way to be tuned against that. Not that their creators would be interested in that, since the LLM's bias is exactly the mainstream opinion that they like very much.

[timmg]

I wasn’t assuming anything. I was asking whether the problem was bias — which we already see in some things that are highly regulated — or just wrong bias.

I’m trying to understand what people think we should correct for.

[contagiousflow]

Correct. They will never not have a social bias. Which leads to the question of, who controls these tools, and what biases are they okay/not okay with specifically training for. Currently they can be seen more as a reflection of broader culture (and even that has problems) but as we're already seeing with Grok they can be tuned at a whim to display any specific ideologies.

[tptacek]

Those are some of the questions it leads to, but there are other questions that situate agency outside of the labs and in the hands of users, like, what processes do you have set up to backstop automated decisionmaking?

It's not interesting to observe that Grok was successfully trained to be an edgelord; anybody paying attention knew that was easily achievable.

[contagiousflow]

> what processes do you have set up to backstop automated decisionmaking?

The companies releasing these models actively encourage the act of automated decision making by them. The entire value proposition is the automation of decisions and knowledge work. It's rare to find a use case for them that isn't offboarding your thinking and therefore agency

[tptacek]

The entire value proposition of the computer industry is the automation of decisions and knowledge work. We are and always have been in the business of automating away people's jobs.

[contagiousflow]

I reckon we agree more than we disagree, but there is a dichotomy of expansive and contractive technologies. Much of the computer industry has given more agency, choice, and knowledge to people.

[tptacek]

That's not in tension with the fact that computers have displaced enormous numbers of jobs. The pitch has always been that the displacement is accompanied by new opportunities elsewhere in the economy.