[wrsh07]

I wish there were more context and maybe the ability to do math on the vectors

Eg what is the real distance between the two vectors? That should be easy to compute

Similarly: what do I get from summing two vectors and what are some nearby vectors?

Maybe just generally: what are some nearby vectors?

Without any additional context it's just a point cloud with a couple of randomly labeled elements

[refulgentis]

You're actually kinda hitting the nail on the head. _Generally_, the word2vec woman + king = queen thing was cute but not very real.

People rarely have to get down to the real true metal on the embeddings models, and they're not what people think they are from their memory of word2vec. Ex. there's actually one vector emitted _per token_, the final vector is the mean. And cosine distance for similarity is the only metric anyone is training for.

In summary, there's ~no reason to think a visualization trying to show multiple vectors will ever be meaningful. Even just starting from "they have way way way more dimensions than we can represent visually" is enough to rule it out

Mini LM v2, foundation of most vector dbs, is 384 dims.

n.b. dear reader, if you've heard of that: you should be using v3! V3 is for asymmetric search, aka query => result docs. V2 is for symmetric search, aka chunk of text => similarly worded chunks of texts. It's very very funny how few people read the docs, in this case, the sentence transformers site.

[tikimcfee]

If I gave you a live GPU shader that let you arbitrarily position any of say a few million words with simple Cartesian coordinates, what would you do with it? Whole words expressed as Individual letters - not symbols, representations, or abstractions. Just letters arranged in a specific order to form words.

[theaussiestew]

I would want to have multi lingual embeddings so I can learn languages more efficiently. Being able to see clouds of different words in different languages would allow me to contextualize them more easily. Same for phrases and sentences.

[tikimcfee]

I’m very much with you. Since most languages I know of are written with combinations of ordered glyphs, they all get rendered the same (although I don’t handle about 20k current Unicode characters of the full 400k+, and none of the grouping really works, so RTL languages would be a mess for individual words).

However, this is exactly where I want to go. A dictionary is a cyclic graph of words mapping to words. That means there’s at least one finite way to visit every single node and give it a position, with a direct relationship to the words that define it, and those words that define them, and so on.

This creates an arbitrary and unique geometric structure per language, and if you get fancy and create modifiers for an individual’s vocabulary, you can even create transforms for a “base” dictionary, and the way someone chooses to use certain words differently. You would be able to see, but likely not understand, the “structure” of types of text - poetry, storytelling, instructional writing, etc.