[datadrivenangel]

Summary: They stopped using vector databases because the performance benefit simply didn't matter compared to how long the LLMs took to respond, and you should focus on using technology to solve problems and not pick the trendy option.

But that never got anyone promoted.

[cbsmith]

You'd be surprised... I've made a pretty good career of identifying cases where simpler, more efficient designs produce better or equivalent results.

[coffeebeqn]

So it seems like they still use vectors - they just replaced the search (however that works) with a dot product operation? I mean from a vector point of view that makes total sense

[dartos]

Searching in LLM land is finding the cosine similarity of two high dimensional vectors. Vector databases try optimizing that operation.

Maybe they found search by plain old dot product faster

[akomtu]

Cosine what? Isn't it just dot product?

[dartos]

There are a few “distance” metrics that are used.

AFAIK cosine similarity or cosine distance is a common one bc it’s faster than a dot product.

[rvz]

More like hype-driven development and magpies making decisions based on how shiny the bracelet is.