Because by being so strong and unkind, it amplifies the recognition of a material or perceptual accident of nature into a needless conceptual flattening of a person's entire presence in the world. Depending on the self-esteem of the person it may be referring to, it's impolite at best, and cruel at worst.
What I think Chris forgets in his writeups is that actual individuals who may fall into those regions of his chart may at some point read it. Why be so abrasive about it?
Ugliness is not a nice thing to have! Like stupidity, or unfriendliness. At the extremes, words for those traits are also generic insults: words like "hideous," "retard," and "asshole" all describe people. And they're often used as more generic pejoratives.
These words are all relative. If you decide that "asshole" is simply too cruel a term, people end up using the next-cruelest term as the cruelest-possible one. So maybe if someone cuts you off in traffic, you say "Jerk!" But now "jerk" can't be used to categorize someone, since it's the worst word.
You'll just be forced to keep making your language more and more bland. There's no logical point at which your argument stops applying: "He's very special, and doesn't play well with others. And he's a more authentically handsome person," will eventually be wordy way to say "He's a hideous, retarded asshole."
There's no loss of meaning, but there's a profusion of syllables. Let's just say what we mean, and assume that people with unfortunate traits have gotten used to hearing about them by now.