[nomonnai]

I believe these issues are instances of the frame problem [0]. Specifying the effects of an action is easy ("show more diversity"), but specifying non-effects is hard to impossible ("do not show more diverse Nazis"). Computer science and logic have worked out how to avoid side effects in formal systems, but the real world is a different animal.

[0] https://en.wikipedia.org/wiki/Frame_problem

[hnfong]

It's more simple (and complicated) than that.

The issues with Google overtuning their AI is the embodiment of the maxim: You just can't please everyone.

It's well known that most formulations of Politically Correct ideologies, when arbitrarily mixed together, aren't necessarily logically consistent. For example, when talking about diversity, generally people talk about portrayal in films for example, but they don't raise the question whether historical figures should receive the same treatment. How about historical figures in a fictional film? (I don't even know the answer to this one, cf. Cleopatra and her historically Greek ethnicity)

Even if you there is theoretically a single correct answer to what is PC in complicated circumstances, there's no way you could encode this information into a language model since the data set is descriptive, and PC is mostly prescriptive. A simple prompt isn't enough, and AI researchers aren't PhDs in contemporary sociopolitical thought.

[rayiner]

I think the bigger problem is that “show more diversity” is a poorly specified mandate. “Diversity” is a narrow concept that arose in the context of college admissions and employment to ensure that people were not being discriminated against on the basis of race. If you say “show me pictures of Harvard students” it’s fair to expect racial diversity in the result. (But you’d probably get that just from the training data.)

But it’s not a concept that makes sense when you generalize it outside the narrow context in which it originates. Especially when your notion of “diversity” is particular to the racial politics of a former slaveholding country that has recently experienced mass immigration from Latin America and Asia. What does “show more diversity” mean in the context of China or Japan? Or even Spanish Harlem? What does it mean in the context of families? Most of the world isn’t diverse, and most of history isn’t diverse, for reasons that are wholly innocuous.

[lsy]

What's amazing to me is that this is the first time I've seen the frame problem explicitly mentioned in online discussion of LLMs, despite it being an incredibly relevant and historically difficult problem in the exact field of AI. Nobody is directly grappling with it or seems to be familiar with the history of the field. The problems the frame problem causes are currently being attacked with efforts to radically expand the context window, but even infinite context doesn't answer the question of what is relevant to an action and what is not.