In more detail https://en.wikipedia.org/wiki/Xenophobia - " It is an expression of perceived conflict between an ingroup and an outgroup and may manifest in suspicion by the one of the other's activities, a desire to eliminate their presence, and fear of losing national, ethnic, or racial identity."
You'd have a (weak) case if the GP had been talking about westerners in general. They were referring to the extensive exploitation of Africa by western powers.
By your logic any criticism against any world event or fact would be xenophobic against those who caused it. That's silly.
Well, yeah, practically textbook:
Popular definition from https://www.merriam-webster.com/dictionary/xenophobia ("Fear and hatred of strangers or foreigners").
In more detail https://en.wikipedia.org/wiki/Xenophobia - " It is an expression of perceived conflict between an ingroup and an outgroup and may manifest in suspicion by the one of the other's activities, a desire to eliminate their presence, and fear of losing national, ethnic, or racial identity."