[softwaredoug]

I feel like we're passing the peak of a vector db hype cycle, where its increasingly clear its one retrieval strategy next to full-text search strategies. I constantly talk to people trying to build RAG and they realize they need a full-text search solution, and a number of strategies, VERY dependent on the task you want your chat system to accomplish.

It's important we get through the trough of disillusionment quickly. There's a lot of market education needed to know when they're truly needed.

[themanmaran]

I fell into this trap as well. Started pretty hyped about vector dbs as the "magical crtl+f". Realized I needed some keyword matching as well. And also some transforms to get the right format for vector search. And also multiple chunking strategies for more fidelity search.

A month in I realize I'm trying to reinvent a search engine. Kinda wonder if I should have just used something like elasticsearch instead.

[Linell]

It's worth saying that ES can work as a vector store itself, so it's very easy to handle a couple different kinds of searches this way.

[finikytou]

full text search is also overhyped. at the end you querying a KB just like in the 90s. the major difference is the scale of the model and the fact that he can make assumptions with a tone that would make you believe what is he saying is a fact

[softwaredoug]

I disagree it’s “overhyped”. I feel like there’s a fairly correct understanding in the market of its uses and limitations. That hype cycle occurred decades ago