John D. Cook has a couple of blog posts about non-intuitiveness of high-dimension geometry [1][2] and this article [3] expands on the very observation that you described.
Quote:
"In his article “An Adventure in the Nth Dimension,” Brian Hayes explores how in high dimensions, balls have surprisingly little volume. As the dimension n increases, the volume of a ball of radius 1 increases until n = 5. Then for larger n the volume steadily decreases."