Your argument comes down to the meaning of "voxel", then. Voxels have volume, and they are represented with polygon faces. If you think that applies here, I guess more power to you.
They don't necessarily need to have polygon faces when you zoom in, they can just be rendered as dots with a single color, as long as adjacent voxels don't show gaps (unlike point clouds). Similar to how there are different methods of zooming into 2D pixel images.