[rspeer]

The machine can learn that someone named "Jamal" is more likely to be associated with the word "perpetrator", particularly in corpora centered on American news.

This is likely a true fact about the world: one that results from racial profiling and unequal enforcement.

It's not desirable to learn that, because encoding this in an AI system's belief about the "meaning" of the name "Jamal" will lead to more racial profiling.

Just because something could be considered "true" doesn't mean it's good to design systems that will perpetuate it being true.