Hacker News new | ask | show | jobs
by dhruv_sagani 1206 days ago
If you are ranking documents based on KNN-style algos against document embeddings, how do you deal with more intentional search queries that have a narrow intent to retrieve a specific type of content?

If I browse around and save content to your app, I might want to retrieve this exact piece of content and not any other one that may be semantically close to the query. My experience with vector-based systems (and let's be honest, we are really talking about a recommendation system rather than a search algo here) is that they are hard to combine in a way that is conducive to efficient information retrieval.

You search for a "dog" and you get "cats" because they are similar, but maybe I really want "dog" and not "cat". I find that tremendously inconvenient when apps force you into semantic search and don't let you choose. Sure that might increase the recall, but still please let me choose.

1 comments

We break the 2 types out in the UI, listing separately. So the user gets to choose. This gives a kind of "best of both worlds" outcome, where a user never gets zero results (and can be make very low fidelity queries), but also doesn't have to sacrifice specificity when the query they're making is one of precision and intentional retrieval.