[_009]

Apart from search, ANNs can be use for recommendations, classification, and other information retrieval problems.

Currently, ES and Solr, both based on Lucene, can't really manage vector representations, as they are mainly based on inverted indexes to n-grams.

ANNs potential applications extend to audio, bioinformatics, video, among any modality that can be represented as a vector. All you need is an encoder! How nice.

Faiss is definitely powerful. I have been running experiments using 80 million vectors that map to legal documents, and vectorizing protein-folds (using Alphafold). While it is an interesting technology, at this moment, perhaps for my usecases, I see it more as a lib or tool than a full-featured product like ES or Solr.

For instance, ATM, updating a Faiss index is a non trivial process, with many of the workflow tools you would expect in ES missing. There is also the problem of encoding the input into vectors, which takes a few milliseconds (do you batch, parallelize, are you ok with eventual consistency?).

I recently been found with pgvector (postgres + vector support) https://github.com/ankane/pgvector. Perhaps less performant, but easier to work with for teams. With support of migrations, ORM, sharing, and all the postgres goodies.

Another interesting/product-ready alternative is https://jina.ai.

And Google's ScaNN, https://www.youtube.com/watch?v=0SvrDtnUgV4

[m-i-l]

> "Currently, ES and Solr, both based on Lucene, can't really manage vector representations"

Lucene does have an ANN implementation due in 9.0, based on HNSW - see https://issues.apache.org/jira/browse/LUCENE-9004 for details. See also https://issues.apache.org/jira/browse/SOLR-12890 and https://issues.apache.org/jira/browse/SOLR-14397 for Solr.

[heipei]

Some more Open Source vector search engines for ANN: https://milvus.io/ https://vespa.ai/ https://vald.vdaas.org/ https://github.com/semi-technologies/weaviate https://opendistro.github.io/for-elasticsearch/features/knn....

[hintymad]

> Apart from search, ANNs can be use for recommendations, classification, and other information retrieval problems.

Yeah, those are all good use cases. I was wondering about a different thing: will the demand for a distributed vector search service concentrate to a few big companies, as smaller companies can use a simpler solution so they don't really need to pay for the technology.

> Currently, ES and Solr, both based on Lucene, can't really manage vector representations, as they are mainly based on inverted indexes to n-grams.

ES has kNN plugin, which stores vectors separately in each segment in Lucene index. Plus, they can also use better storage formats and algorithms.

[rpedela]

Google has a distributed embedding matching service in preview: https://cloud.google.com/vertex-ai/docs/matching-engine/over...

I guess it depends on what you mean by "simple". The algorithms are complex but there are good tools that implement them. I would imagine smaller companies would use off the shelf tooling, and I would argue that is simpler. Vector embeddings are so unbelievably powerful and often yield better results than classical methods with one of the good tools + pretrained embeddings.

Specifically for search, I use them to completely replace stemming, synonyms, etc in ES. I match the query's embedding to the document embeddings, find the top 1000 or so. Then I ask ES for the BM25 score for that top 1000. I combine the embedding match score with BM25, recency, etc for final rank. The results are so much better than using stemming, etc and it's overall simpler because I can use off the shelf tooling and the data pipeline is simpler.

[hintymad]

> I match the query's embedding to the document embeddings,

I assume the doc size is relatively small, otherwise a document may contain too many different topics that make it hard to differentiate different queries?

[rpedela]

For my search use case, documents are mostly single topic and less than 10 pages. However I have found embeddings still work surprisingly well for longer documents with a few topics in them. But yes, multi-topic documents can certainly be an issue. Segmentation by sentence, paragraph, or page can help here. I believe there are ML-based topic segmentation algorithms too, but that certainly starts making it less simple.

[gk1]

The moment you cross 10M items or 100 QPS then scaling such a system becomes non-trivial. That's not a high threshold for any enterprise software company handling customer data or any consumer tech company with >10M users. Once you add other requirements to the mix, such as index freshness and metadata filtering, the managed options where this is already built-in start to become compelling even at lower volumes.

Also, Pinecone (disclosure: I work there) has usage-based billing that starts at $72/month, so "paying for the technology" is not that scary.

[sanxiyn]

Well, OpenSearch already has ANN search plugin. Implementation currently uses nmslib. I am using it and it works fine.

https://opensearch.org/docs/search-plugins/knn/index/

[jjgreen]

I've been looking at Faiss, it's on its way to Debian ...

https://packages.qa.debian.org/f/faiss.html