So I have 1.6M rows of title, body and author. Can I use SDDB to embed all rows with a local model and then do semantic search as a query? And how can this scale past a single machine?
Absolutely you can query the embeddings of 1.6M rows, you can try with lance or in memory vector search type.
The scalablity will depends on embedding size, machine configurations, etc.
Thanks
Absolutely you can query the embeddings of 1.6M rows, you can try with lance or in memory vector search type. The scalablity will depends on embedding size, machine configurations, etc. Thanks