| I meant that if you say "there's no way to avoid the bias", it sounds like you're basically admitting defeat and not even trying to reduce bias. I don't think we can achieve 0% bias, I agree with you on that. But I think that, if you decide to spend some amount of effort (let's call this amount "X"), it is possible to reduce bias compared to if you spent zero amount of effort. And that furthermore, if you spend a "Y" amount of effort where X<Y, then you can reduce even more bias. Obviously, at a certain point this would have diminishing returns, so presumably there is some sweet point where, even though you can't be 100% unbiased, you can at least say you made a reasonable effort to be unbiased and that your remaining sources of bias are unintentional (and probably, almost just as likely to go in one direction vs another). To bring the conversation back to the original topic, I think ChatGPT/InstructGPT is actually being actively biased towards one political side as a side effect of RLHF being done by people from OpenAI, even if this bias is being introduced unintentionally. It would be much better, for example, if OpenAI could somehow accomplish RLHF using a sample of its users as the AI trainers. It would still be a far cry from reaching 0% bias asymptotically, but it would already be an improvement, I think, as I think its users are a more representative sample of the population than OpenAI employees. |