Still shows implicit bias and you know structural racism. It is actually looks like it could be a great measure of structural racism since the LLM is trained on everything...
But if for example my native language which is spoken by far fewer people than AAVE or Standard American English was given to LLMs, which LLMs also frequently fail at - especially anything besides GPT-4, and try to correct something that actually is correct, wouldn't it also naturally if it's forced to judge it rate it as more gibberish?
I am asking questions and sharing thoughts about the research I saw above. I do not have the funds, time or motivation to do this type of research.
Just my first thought about the article is that it is doing this "think of the children" type of thing to make a case for recalling something that is really useful for many people.
Yes, also to be clear, I am not from US and neither am I a native English speaker, but to me it's obvious that in professional or business context a person must put effort in to have seemingly clear and formal English, unless they are known as this highly skilled, experienced and an eccentric character so they could potentially pull off writing in however they please.
So I would imagine that any deviation of this expected norm would yield in a result that might be perceived as more "stupid or lazy", whether it's street dialect, US southern, redneck dialect, teenager dialect, mix with other languages or otherwise.
So my point is - that it's measuring a correlation from the standard, professional, business English, rather than it being anything to do with racism.