[im3w1l]

There are lots of perspectives on vectors and tobinfricke gives one. Let me give another.

Given a coordinate system, a vector can be represented by a tuple of numbers, just like a point can be represented by a tuple of numbers. The point p, and its position vector i.e. the vector from the origin to p will have the same tuple of numbers. The magnitude of the vector corresponds to the distance of p to the origin. So points or vectors, well it's just a choice of words without a material difference.

If you use the word vectors then you do kind of sort of imply that you could do the vector operations, scalar-multiplication, and vector addition, and getting something semi-useful out. This is indeed sometimes done with embeddings. But most of the time it's in the form of an affine combination (weighted sum where weights sum to 1) which is something you can do for points too.

Edited a bit