Hacker News new | ask | show | jobs
by farleykr 705 days ago
What’s interesting to me is that (if I understand correctly) they’re using GPT as a source of information to make this claim. Usually you see conclusions like these being drawn from some sort of sociological study. But here they’re just talking to GPT and using its answers to determine things about the real world.

Not disputing the claims. But talking to GPT to get answers about the real world that have to do with value judgements is just weird. There’s a big difference between asking GPT to give you a recipe for a cake and asking GPT to help you understand the value the world places on different people.
3 comments
ath92 705 days ago
The paper is about biases in GPT-4. They're not talking to GPT-4 to determine things about the real world. They're talking to GPT-4 to determine things about GPT-4.
link
farleykr 705 days ago
> This phenomenon likely reflects that while initiatives to integrate women in traditionally masculine roles have gained momentum, the reverse movement remains relatively under developed.

Not challenging you. Maybe it’s just the phrasing. But that sentence to me reads as if they think the presence of the biases in GPT means that they exist in the real world. And again, not challenging that the biases do exist. Just noting the trend toward trusting GPT in increasingly subjective areas that have to do with moral judgement. To me it’s not too much different than drawing conclusions about the world from religious texts.

link
sokoloff 704 days ago
I read as suggesting that the underlying bias is contained in the text corpus that GPT-4 was trained on, which in turn suggests it was put there by the humans who authored the original corpus.

That seems an entirely plausible mechanism to me.

link
ajsnigrutin 705 days ago
But GPT 'learns' from the world. GPT did not sit on the toilet and think about it out of the blue, GPT analyzed "the world" (well,... the written part on the internet) and came to these conclusions and biases.
link
tossandthrow 705 days ago
The OpenAI models are also aligned on top of their base training on order to have them behave in a certain way.

Therefore it is not really a proxy for the real world.

link
tossandthrow 705 days ago
Yes, this is not sound and constitutes severe critism of the work.

ChatGPT is heavily aligned in order to reduce What society sees as biases.

This study likely just reverse engineers some of these alignments.

link