Yep, sadly. There are DEI programs which are predicated on overt racism and sexism, including redefinition of the term racism so that it is impossible by definition to be racist towards some races, redefining being race blind or neutral as actively racist (leading, for example, to the recently failed California Prop 16), and establishing the position that the only way to be non-racist is through opposing discrimination. You can't go by the title of something, just like -- you know, the "Patriot Act" was really anything but patriotic.
Bringing it back to the subject at hand: With both Google and OpenAI we've seen "machine learning fairness" initiatives that seek to counter perceived biases in results (which are biases existing in the real world and/or training material, when they're even biases at all) by adding explicitly discriminatory optimizations.
Explicit examples include OpenAI augmenting user prompts to require that the output be "black" or "female" (but not other sexes/races, and to the detriment of the results quality regardless): https://twitter.com/rzhang88/status/1549472829304741888 (also pretty ignorantly even by their own goals, considering that the change made it even more likely to produce black people for 'prisoner' or 'convict' even though it was already very likely to do so)
Similarly, google image search used to return mostly white men for "CEO" which, while unfortunate, reflected the underlying material. Today, for me when I do the search every person in the first screen of results is a woman or dark skinned. A search on bing image search gives results more similar to what Google used to give: e.g. still over-representing women compared to the profession, but probably similarly to coverage on the internet. And we know from secret recordings and leaked documents that this isn't some random quirk-- it was an intentional change intended to effect positive social change.
The fact that these intentional counter biases are performed in secret, cannot be disabled by users, are inherently highly subjective, and almost inevitably reduce the quality of the results by any metric that doesn't include the social/political goals should be a concern for anyone who's only access to these powerful ML tools is remote access to a black box.
I don't want to argue that laying a thumb on content generational machine learning to produce more intersectional results is some kind of crime against humanity. It's clearly an attempt made with good intentions, but the greatest of evils are usually performed by someone with good intentions. Explicitly using adjustments which are pro some races and anti-others is something we ought to be concerned about, especially when it's done in secret and is non-optional.
A fundamental challenge is that these modern ML tools are largely application agnostic. In some applications injecting the right kind biases is neutral or beneficial, in others it's actively harmful. One of the things I've found large language models and image generation models useful for is sampling the biases in the underlying training data-- to find out what kind of secondary meaning might exist in the words I use in my writing, to learn that a word that I was going to use also carries some unintentional overtones or acts as a dog whistle (racial, sexual, political, etc.) in a manner I wasn't aware of. "Fairness" hacking the results undermines this usage by substituting biases in the training set with the preferences of some publicly unaccountable staff in the organization that controls the ML model.
I think that the best anyone can do for application agnostic models is to match the biases of the model to the training material and disclose what the training material is and the known biases in them, and provide optional counter-biases (with disclosed properties) if there is user demand but clearly the direction at these firms is otherwise: You get the augmented model and they argue that the public shouldn't even be permitted access to the training-reflecting model, even calling them "unsafe".
I’ve worked in tech. The overt bias observed in the common worker (male dominated) is palatable and is undoubtedly interjected into the work.
You’re dealing with a technology that will have an impact on everyone everywhere.
If you don’t want to include them in the development, and are unable to police yourselves on addressing the inherent biases, then I don’t think you can complain.
Bringing it back to the subject at hand: With both Google and OpenAI we've seen "machine learning fairness" initiatives that seek to counter perceived biases in results (which are biases existing in the real world and/or training material, when they're even biases at all) by adding explicitly discriminatory optimizations.
Explicit examples include OpenAI augmenting user prompts to require that the output be "black" or "female" (but not other sexes/races, and to the detriment of the results quality regardless): https://twitter.com/rzhang88/status/1549472829304741888 (also pretty ignorantly even by their own goals, considering that the change made it even more likely to produce black people for 'prisoner' or 'convict' even though it was already very likely to do so)
Similarly, google image search used to return mostly white men for "CEO" which, while unfortunate, reflected the underlying material. Today, for me when I do the search every person in the first screen of results is a woman or dark skinned. A search on bing image search gives results more similar to what Google used to give: e.g. still over-representing women compared to the profession, but probably similarly to coverage on the internet. And we know from secret recordings and leaked documents that this isn't some random quirk-- it was an intentional change intended to effect positive social change.
The fact that these intentional counter biases are performed in secret, cannot be disabled by users, are inherently highly subjective, and almost inevitably reduce the quality of the results by any metric that doesn't include the social/political goals should be a concern for anyone who's only access to these powerful ML tools is remote access to a black box.
I don't want to argue that laying a thumb on content generational machine learning to produce more intersectional results is some kind of crime against humanity. It's clearly an attempt made with good intentions, but the greatest of evils are usually performed by someone with good intentions. Explicitly using adjustments which are pro some races and anti-others is something we ought to be concerned about, especially when it's done in secret and is non-optional.
A fundamental challenge is that these modern ML tools are largely application agnostic. In some applications injecting the right kind biases is neutral or beneficial, in others it's actively harmful. One of the things I've found large language models and image generation models useful for is sampling the biases in the underlying training data-- to find out what kind of secondary meaning might exist in the words I use in my writing, to learn that a word that I was going to use also carries some unintentional overtones or acts as a dog whistle (racial, sexual, political, etc.) in a manner I wasn't aware of. "Fairness" hacking the results undermines this usage by substituting biases in the training set with the preferences of some publicly unaccountable staff in the organization that controls the ML model.
I think that the best anyone can do for application agnostic models is to match the biases of the model to the training material and disclose what the training material is and the known biases in them, and provide optional counter-biases (with disclosed properties) if there is user demand but clearly the direction at these firms is otherwise: You get the augmented model and they argue that the public shouldn't even be permitted access to the training-reflecting model, even calling them "unsafe".