[Scene_Cast2]

Well, depends. For some models (especially two tower style models that use a dot product), you're definitely right and it makes a huge difference. In my very limited experience with LLM embeddings, it doesn't seem to make a difference.

[extasia]

Interesting, I hadn’t heard of two tower modes before!

Yes, I guess it’s curious that the information lost doesn’t seem very significant (this also matches my experience!)

[Scene_Cast2]

Two tower models (and various variants thereof) are popular for early stages of recommendation system pipelines and search engine pipelines.