|
|
|
|
|
|
by westoncb
3314 days ago
|
|
|
I think I get part of the N-ball analogy, but I'm trying to piece together the rest... So it seems like one problem/feature with high dimensional spaces is that you have more regions which are orthogonal to one other. So, it's possible to have quite a bit of motion, which to a human mind might give the impression that one is doing quite a lot—but then you might discover a new way of turning around and find that space extends in another direction, too, at which point you realize you covered a smaller fraction of the available space than you previously believed. What I don't get is how the volume of an N-ball with fixed radius going to zero as N goes to infinity plays into the analogy. When you say, "when you compare it to a hypercube of the same dimensional count," I don't get what in the programming analogy would link to the N-ball rather than the high dimensional hypercube. I also get the algebraic justification for this property of N-balls, but I still have no real geometric intuition for why it would be the case. Could you clarify what's meant by, "as the number of dimensions increases the closer the 'halfway' point of the volume of the N-ball gets to the edge of the N-ball"? Not sure what's meant by 'halfway' point here, or edge of the N-ball. My guess is that you're talking about distribution of volume, like if you have a tapered cup that gets small at the bottom, most of the volume will be at the top. |
|
|
Factorials grow much faster than exponents, so with an N-Ball we expect the "volume" to be small if we were to compare it to the corresponding hypercube when the dimension and when the diameter / length are all equal. After all we're comparing approximately diameter * R ^ D / D! and length * R ^ D. So if you put an N-Ball into a hypercube with length = diameter, what you'll discover is that the N-Ball is tiny when compared to the hypercube. Which means we have the weird and counter intuitive conclusion that all the "volume" of a hypercube is in the corners. Try calculating the volumes yourself using a high dimension like 100. Even better plug the whole thing into a script so you can generate a table and see the effect get progressively worse as you increase the dimension count.
There is also a similar effect on the N-Ball itself. Imagine drawing a smaller circle inside of a circle where their centers are the same such that the inner circle contains exactly half of the area of the outer circle. Now imagine doing this in a sphere. Now a 4D sphere. Now a 5D sphere. What you'll find is that as you increase the dimension count, the radius of the inner circle will get longer and longer. Which means that half of the "volume" of an N-Ball slowly moves closer and closer to the outer edge of the N-Ball as you increase dimensions.
Here's the punch line. Imagine that the N-Ball represents all there is to know about a topic. You begin in the center of the N-Ball and in order to learn more you have to walk to the outer edge of the N-Ball. Also imagine that you're only able to see what's right in front of your face. In the beginning at the center there is very little volume to look at. It will appear to be easy to understand what's going on. But as you walk to the edge of the N-Ball, the amount of volume that you're able to see increases. When you get right to the edge of a very high dimension N-Ball you will notice that you only know half of what is available to know.
Simple topics are N-Balls with low dimensions. It's easy to "walk" your way to knowing half of everything there is to know. Complex topics are N-Balls with high dimensions. Programming (and by extension software engineering) is really high dimensional. The more you learn the more you notice there is to learn. When you walk to the edge of what seems possible you find yourself swimming in a sea that dwarfs you. But in the beginning it looks like there isn't really all that much going on.
[1] - https://en.wikipedia.org/wiki/Volume_of_an_n-ball#The_volume