Hacker News new | ask | show | jobs
by leobg 1229 days ago
Slight overkill to use GPT, though it works for the author and I can see that it’s the low hanging fruit, being available as an API. But this can also be done locally, using SBERT, or even (faster, though less powerful) fastText.

Also, it’s helpful not to cut paragraphs into separate pieces, but rather to use a sliding window approach, where each paragraph retains the context of what came before, and/or the breadcrumbs of its parent headlines.
2 comments
cratermoon 1228 days ago
I just put my notes into ElasticSearch and use the TextAnalysis processor and get a lot of the same functionality.
link
alostpuppy 1228 days ago
This is interesting. How effective was this?
link
cratermoon 1228 days ago
ElasticSearch's text analyzer isn't semantic, but it does stemming and a few other transformations. Combined with the ranking of results by match, I can type in a couple of words or short phrase and get back a reasonable set of notes. That combined with the tags I used and linking means I can typically find the most relevant notes pretty quickly. It might be nice to have some ability for it to match synonyms of search terms given, eg. 'movie' and 'cinema'. I worry about overzealous collapsing of synonyms, for example 'film' can mean 'movie', but mostly they are different concepts.
link
dchuk 1228 days ago
When using SBERT instead of gpt for this use case, is it paired with some sort of vector database or just all done in code/memory?
link
leobg 1228 days ago
You’d want persistence, since the embedding process takes some time. But you don’t need to go all Pinecone on this. There is FAISS, and there is hnswlib, for example. Like SQLite for vector search.
link
gk1 1228 days ago
Friendly reminder that we (Pinecone) have a free tier that holds up to ~5M SBERT embeddings (x768 dimensions). For quick projects, going "all Pinecone on this" could turn out to be the easier and faster option.
link
leobg 1228 days ago
Point taken ;-)

I like to stand up for the little guy. I hear Pinecone this and Pinecone that. And nobody seems to pay any attention to the awesome dude who made hnswlib.

link
gk1 1228 days ago
Who, Yury Malkov? He won’t be offended… He’s an advisor to Pinecone. :)

And yes, both he and HNSW are awesome.

link
moneywoes 1228 days ago
What about Postgres with pg vector?
link