The concept of “one” holds a dual role. It represents a countable unit: something you can put in a bowl and also stands for indivisibility itself. When you divide any quantity by an indivisible unit, you’re simply counting how many of those indivisibles fit within it. Then comes 2: the first number that is divisible, but only by itself and the indivisible one. That’s what makes it prime. A prime is a number divisible only by itself and by 1, the indivisible origin of all counting.
Your explanation is true of every prime. I’m pretty sure GP just meant that “2 is the only prime with the additional characteristic of being an even number”. So it’s odd (read “interesting”) in that sense, like if it would be if (for example) any number were to be the sole prime composed of exactly X digits.