[vineyardmike]

Alternatively, why should toys make the person playing with it feel bad? Toys are intentionally escapist, it’s the point? Feels like a weird argument.

I think the better comparison is what the other commenters brought up - maps. These are tools, and we turn to them for a purpose. And yet even geography is not ahem set in stone. We need our tools to be useful to the user, and that means different things to different people. There’s dispute in all facts. Just ask the people living in Taiwan, or Gaza or Ukraine…

[gotoeleven]

That one AI calls it taiwan and another calls it china's extra cool eastern china island doesn't seem very interesting to me. There have always been and always will be land disputes. What is more interesting is, suppose you trained an AI with the objective of minimizing "harm" (in the sense of hurting people's feelings that everyone uses now) rather than minimizing error with the truth?

[vineyardmike]

Is it harmful to tell the user that Taiwan is the independent nation of Taiwan? It’s “not true” (depending on who you ask) and could “hurt the feelings” of many people. I bet if you went to mainland china and went to Taipei and talked about this, you could offend people with your version of “minimizing error”. There is no way to please both parties at once, one side will get hurt, and there is no “truth”.

That “little girl from Harlem” will never be a mid-evil princess, but neither will literally anyone alive today, because princesses and castles are a thing of the past. Seems like an example meant to straw man a cultural-war viewpoint. That little girl may factually become a doctor. Should the AI show pictures of (grown up versions of) little black girls from Harlem when asked to generate a picture of a doctor?

The culture war outrage over contrived examples in the name of “maximal truth” always misses that the attempts to showcase minorities and counter bias is adding truth to the output. Just because a bias is being artificially suppressed doesn’t mean that the bias was truthful. Sometimes it makes mistakes, but that would still be true if no “corrective” action was applied.

[corimaith]

At a certain point we do need to go back into the basics of why we are having discussion in the first place.

Taiwan de jure might be ambiguous, but de facto it more or less is a country. If you were working out a trade deal, booking a holiday, immigrating, etc, if you actually needed to know then treating Taiwan as part of China would run into so many inconsistencies it would be impossible to function.