[kpango]

Since Vald is a pure ANN search engine, it does not support graph search of boolean filtering results. However, it does have post-filtering capabilities. The same use case can be achieved by using the post-filter. It is possible to implement functions such as retrieving more TopKs than the expected value of N, and then filtering the results to get N in the end. The documentation for Vald's Filter feature is in preparation, but should be available soon. There are also functions that work seamlessly with Tensorflow and ONNX, so it is possible to embed text data in vector space without Boolean filtering, and perform similar searches using only ANNs by weighting vector queries. Our ideal world would be one in which everything becomes a vector, and we could perform a variety of searches using vectors.

[hintymad]

thanks. The challenge with post-filtering is that the K in TopK becomes unpredictable if I'd like to get a fixed page size. I may end up issuing multiple queries with progressively large Ks to get a single page of results.

[etiennedi]

Exactly. See my reply on this comment thread comment thread to see how Weaviate solves the filter-issue by using an inverted index to produce a whitelist of IDs which is then passed to the vector index where non matching IDs are simply skipped.