[tourist_on_road]

Great work! Looks like you are using HNSWLIB. From what I understand HNSW graph based approach can be memory intensive compared PQ code based approach. FAISS has support for both HNSW and PQ codes. Any plans on extending your work to support PQ code based index in future?

[cyrusthegreat]

Yes! We plan to bring Faiss in and utilize a lot of its functionality, our goal for this release was to get an end-to-end working to get feedback on the API. HNSW was a good default with this in mind.

[jamesblonde]

How does it compare to the OpenDistro for Elastic KNN plugin - which also uses HNSW (and also includes scalable storage, high availability, backups, and filtering)?

[cyrusthegreat]

Our API is built from the ground up with the machine learning workflow in mind. For example, we have a training API that allows you to batch requests and even download your embeddings and generate an HNSW index locally. Our view of versioning, rollbacks, and more makes a lot of sense for an ML index, but very little sense for a search index.