|
|
|
|
|
by wolfgangK
963 days ago
|
|
About words embeddings, the №1 example is the famous King - Man + Women = Queen
This works nicely in the vector space but fails to make a visual impression when projected on 2 dimensions. Neither with ACP, nor MDS ot t-SNE in my experience :
https://bhugueney.gitlab.io/test-notebooks-org-publish/jupyt... (← JupyterLite Notebook doing words embedding in the browser : don't try to run this on a smartphone !) Does anyone know how to nicely visualize the poster child of words embeddings ? |
|
In 3d you can go one further and have the Z-axis be "king"-"queen" (or gram-schmidt again). The orthogonalized versions have the advantage that they give a closer notion of distance to what the underlying model sees. In the 2d case you will get exact distances except that it won't show how far off "queen" you are when you compute "king"-"man"+"woman". In the 3d case it should give exact distances.
Edit to add: With the 2d version you can maybe do some more stuff. IIRC "queen" is chosen as it's the word with the closest embedding to X="king"-"man"+"woman". You can put the next few closest words on the 2d chart as well, each labeled with the orthogonal distance from the 2d plane. So then "queen" should be the word with the smallest (squared distance from X) + (squared orthogonal distance from plane), which you might be able to eyeball.