That's just a figure of speech. The charge of an ion stems from the ratio of electrons to protons. To every "hole" there is a proton, which is a particle. I am repeating myself.
It's not just a figure of speech, it's a useful abstraction. The protons don't move, but the holes seem to do that. Not only that, if you pretend the holes have a certain amount of mass you can predict how they move, and they move similarly to how a positibly-charged particle would move in the same circumstances.