The AI is very honest and innocent, it doesn't know what political correctness is. I've heard stories of parents whose kids would also mislabel a black human as a gorilla.
An 'innocent mistake' performed by a megacorporation with bajillions of dollars, of exactly the same kind of error that has publicly appeared in the actions of other megacorporations with bajillions of dollars... at that point it's not a 'mistake', it's just the people in charge not caring.
AI doesn’t build itself in a vaccuum. The very people who train it are likely to be biased in one direction or another and if left unchecked their mastetpiece will be biased as well. Now, if these algos were staying in a lab or something it wouldn’t be a problem but as soon as they hit the real world they should be held up to some standards, don’t you think?