[OmarShehata]

Are you saying it's pure chance that operations like "man - woman" = "king - queen" (and many, many other similar relationships and analogies) work?

If not please explain this comment to those of us ignorant in these matters :)

[StrangeDoctor]

It’s not pure chance that the above calculus shakes out, but it doesn’t have to be that way. If you are embedding on a word by word level then it can happen, if it’s a little smaller or larger than word by word it’s not immediately clear what the calculation is doing.

But the main difference here is you get 1 embedding for the document in question, not an embedding per word like word2vec. So it’s something more like “document about OS/2 warp” - “wiki page for ibm” + “wiki page for Microsoft” = “document on windows 3.1”

[napoleongl]

3Blue1Brown has some other examples in his videos about transformers, most notably I think is that hitler-Germany+italy ~ Mussolini!

https://www.3blue1brown.com/lessons/gpt