|
|
|
|
|
|
by simonw
1008 days ago
|
|
|
"There's three key metrics we track. CREATE INDEX time, SELECT throughput, and SELECT latency." There's a fourth metric that I'm really interested in: assuming it's possible, how long does it take to update the index with just one or two updated or inserted vectors? Is the expectation with this (and the other) tools that I'll do a full index rebuild every X minutes/hours, or do some of them support ongoing partial updates as data is inserted and updated? Just had a thought: maybe I could handle this case by maintaining an index of every existing vector, then tracking rows that have been created since that index itself. Then I could run an indexed search that returns the top X results + their distance scores, then separately do a brute-force calculation of scores just for the small number of rows that I know aren't in the index - and then combine those together. Would that work OK? Even if the index doesn't return the scores directly, if it gives me the top 20 I could re-calculate distance scores against those 20 plus the X records that have been inserted since the index was creation and return my own results based on that. |
|
|
Here’s a chart for INSERT latency (sorry about the formatting): https://docs.lantern.dev/graphs/insert.png
At the moment, we underperform Neon wrt this metric, but a better implementation is coming soon that will address this.
> Is the expectation with this (and the other) tools that I'll do a full index rebuild every X minutes/hours, or do some of them support ongoing partial updates as data is inserted and updated?
The HNSW algorithm updates the index after every insert. So all existing HNSW options (Lantern, pgvector, Neon, …) already support this.
With pgvector IVFFlat, you expect the performance to degrade over time, and you will need to re-index. This is because IVFflat’s index quality heavily depends on the centroids chosen at index creation time. HNSW does not have this limitation.
In both cases, you might want to do a full-index build to tune your hyperparameters.
We’re working on this in a few ways. One is automatic hyperparameter tuning. Another is supporting external index creation that would offload this to another server. Does this answer your question?