[jameshart]

You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers, which we sometimes interpret geometrically because it helps us comprehend them. High dimensional vectors grow increasingly ungeometric as we have to wrestle with increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’.

In the end, vectors (long lists of numbers a1, a2, a3, … an) start looking more like discrete functions f(i) = ai. And you can extend the same concept all the way to continuous functions - they’re like infinite dimensional vectors. For continuous functions over a finite interval the dot product (usually called the inner product in this domain) is just the integral of the product of two functions, and the ‘magnitude’ of a function is its RMS, and that means functions have a ‘cosine similarity’ which is not remotely geometric. There isn’t any geometric sense in which there is an ‘angle between’ cos(x) and sin(x) except it turns out that they have a cosine similarity of 0 so it implies the ‘angle between’ them is 90°, which actually makes a lot of sense. But in this same sense there’s an ‘angle between’ any two functions (over an interval).

But we are not doing geometry here.

[bowsamic]

> You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers

No. They can be expressed as lists of numbers in a basis if the vector space is equipped with a scalar product but the vector itself is an object that transcends the specific numbers it is expressed in.

What you’re saying here is totally wrong and I recommend you check out the Wikipedia page on vector spaces. The geometrical object “a vector” is the more fundamental thing than the list of numbers

[xigoi]

Tuples of numbers are a special case of a vector space, which even comes with a canonical basis and inner product for free. And since the article is about word embeddings, which map words to tuples of numbers, there’s no need to mention other vector spaces in this context.

[jameshart]

You think this comment could have been written by someone who doesn’t understand what a vector space is?

Vectors are not purely geometric objects. Geometry is a lens through which we can interpret vectors. So is linear algebra. The objects behave the same and both perspectives give us insights about them.

Insisting vectors are only geometric is like saying complex numbers are geometric because they can be thought of as points on the complex plane.

[drdeca]

> increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’.

Implausible how? “geometric” doesn’t mean “embeds nicely in 3D space”.

What’s wrong with talking about the angle between two L^2 functions defined on an interval? Geometric reasoning still works? If you take a span of two functions, you have a plane. What’s the issue?

[HPsquared]

In this case can people just prepend "hyper-" as in hyperplane etc? Hyper-line, hyper-angle. (Speaking as someone who has heard 'hyperplane' a few times but not others)

[klodolph]

No, that would be incorrect. A plane is 2D. If you have two functions, and take their span, you get a 2D plane. It is a regular, flat, 2D plane.

When people say “hyperplane” they are generally talking about something with more than two dimensions.

[drdeca]

At least when the ambient vector space is more than 3-dimensional, yeah. Specifically, a hyperplane generally refers to something with codimension 1.

(So, when the ambient vector space is finite-dimensional, the dimension of a hyperplane is one less than the dimension of the ambient vector space.)

[klodolph]

> that means functions have a ‘cosine similarity’ which is not remotely geometric.

It obeys the normal rules you learned in geometry. For example, pick three functions a,b,c. The functions form a triangle. The triangle obeys the triangle inequality—the distances satisfy d(a,b) ≤ d(a,c) + d(c,b). The angles of the triangle sum to 180°.

This sounds an awful lot like geometry to me.