[_vxw6]

I’m adding some technical details!

Haystack runs entirely client sided in the browser, so it has a unique tech-stack:

Storage

using IndexDB, haystack stores user indexes locally, + a compressed 90mb NLP model (t5-small) is stored.

Indexing

Locally in the browser, using a t5-small bi-encoder, and some parsing of documents happens in wasm.

Search

Query converted to embedding, then searched over index, atlast results are reranked with a t5-small based cross encoder, and top results go through a seq-to-seq transformer to produce a nice consise textual answer.

[bsima]

Very cool tech stack. How do you run t5-small in the browser?

[_vxw6]

I load the 90mb model into memory from IndexDB, and then some rust code compiled down to WASM does calculations with the model.