[pallas_athena]

An Embedding is a n-dimensional vector (think of it as a sequence of n numbers).

During training, each token (or word) gets an Embedding assigned.

Critically, _similar words will get similar embeddings_. And "similar words" could mean both semantically or (as was the example) syntactically ("apple" and "appli").

And being vectors, you can do operations on them. To give the classic example, you could do: Embedding(`king`) + Embedding(`female`) = Embedding(`queen`).

[pertymcpert]

It's completely mind blowing that you can add the vectors like that.