I kind of agree. I don't know how the process works which is why I'm asking for someone more knowledgeable. But my current understanding is that is how it is done.
Or pretend to have lost their passport and get a new passport without stamp. Or get a secondary passport that's made available by most governments because of issues like this one.
But you are right in principle. Letting people through because they look important is a big security hole.