I don't understand why you're saying that - it's contrary to the nature of symbols and language. Symbols are a unit of meaning, so their usage dictates their meaning. If Pepe was used exclusively by the right, then it would be a right wing symbol. If it was used by various people, but was particularly popular with the right to express right wing ideologies as memes, even then it would still be a right wing symbol when used as such. Symbols very frequently have multiple meanings, sometimes where one is dominant.
Anyway, I'm not commenting on where I think reality lands, just on your comment.
This symbol is (probably) mostly used by certain small group of right wing people, but it is not used for self identification (unlike swastika by nazi).
So swastika is a nazi symbol, but pepe is not.
Or another example: Star of David is often used by ZOG conspiracy theorists, but it does not make Star of David ZOG symbol.
Anyway, I'm not commenting on where I think reality lands, just on your comment.