Centre Point and it's lesser known baby brother One Kemble Street are pretty attractive buildings too though the former has the characteristic brutalist issue of not being great at street level. Depending on where you approach it from, the Barbican can have that issue too...
Note that he did not use the expression you are criticizing him for misusing.
The rule is the rule, and exceptions are the exception. Exceptions do not make the rule, by definition, so if your only defense of Brutalism is to say 'look at this one exception out of the tens of thousands that got built, which doesn't suck!', then you have conceded the point about Brutalism sucking.