[ww520]

The vector here refers to the "feature vector" where the dimension is the number of elements in the vector. E.g. a feature vector of [size, length, width, height, color, shape, smell] has 7 dimensions. A feature vector for the space has 3 dimensions [x, y, z]. The term "higher dimension" just means the number of features encoded in the vector is higher than usual.

In the context of neurons, while the neurons are in the 3 spatial dimensions, the connections of each neuron can be encoded in a feature vector. Each connection can specialize on one feature, e.g. the hair color of the person. These connection features can be encoded in a vector. The number of connections becomes the dimension of the vector. Not to be confused with the physical 3D spatial dimensions of the neurons.

The nice thing about encoding things in vectors is that you can use generic math to manipulate them. E.g. rotation mentioned in this article, orthogonality of vectors implies they have no overlap, or dot product of vectors measures how "similar" they are. Apparently this article shows that different versions of the sensory data encoded in neurons can be rotated just like vector rotation so that they are orthogonal and won't interfere with each other.

Linear algebra usually deals with 2 or 3 dimensions. Geometric algebra works better on higher dimension vectors.