I've heard the same by some individuals who say they don't like people of color. Perhaps your line of reasoning is not complete.