Can you say more? Usually projects that gravitate to SQLlite are not those that require massive scale and a FAISS index of a few GB covers a lot of documents.
My dataset is going to be around 10M documents. With OpenAI embeddings, that will be around 62GB. AFAIK SQLite should be able to handle that size, but I haven't tried.
This is not going to be my primary DB. I would update this maybe once a day and the update doesn't have to be super fast.
This is not going to be my primary DB. I would update this maybe once a day and the update doesn't have to be super fast.