[kevincox]

I hadn't had much luck with any of these. Phind is indeed better for programming but it still has the same trouble with the obscure stuff.

It seems that the main problem right now is that they only look at the first few results. Basically they all do this:

1. Ask model for a search query.

2. Run search and feed the results into the model.

3. Ask model for an answer.

The problem is that for very specific questions the search results won't contain the required information. So it has to say something irrelevant or hallucinate. Especially for negative queries like "a foo library that doesn't depend on std" or similar it really struggles because it can't effectively filter out libraries that fail the requirements and can't keep searching until it finds one.

Basically they can be fast if a decent query can have the answer within the first page of results. But fail otherwise. In many cases you are better off just reading through search results on your own.