[Anamon]

I see these approaches a lot when I look over the shoulders of LLM users, and find it very funny :D you're spending the time, effort, bandwidth and energy for four carefully worded questions to try and get a sense of the likelihood of the LLM's output resembling facts, when just a single, basic query with simple terms in any traditional search engine would give you a much more reliable, more easily verifable/falsifiable answer. People seem so transfixed by the conversational interface smokeshow that they forgot we already have much better tools for all of these problems. (And yes, I understand that these were just toy examples.)

[Zambyte]

The nice thing about using a language model over using a traditional search engine is being able to provide specific context (ie disambiguate where keyword searches would be ambiguous) and to correlate unrelated information that would require multiple traditional searches using a single LLM query. I use Kagi, which provides interfaces for both traditional keyword searches, and for LLM chats. I use whichever is more appropriate for any given query.

[tguedes]

It really depends on the query. I'm not a Google query expert, but I'm above average. I've noticed that phrasing a query in a certain way to get better results just no longer works. Especially in the last year, I have found it returns results that aren't even relevant at all.

The problem is people have learned to fill their articles/blogs with as many word combinations as possible so that it will show up in as many Google searches as possible, even if it's not relevant to the main question. The article has just 1 subheading that is somewhat relevant to the search query, even though the information under that subheading is completely irrelevant.

LLMs have ironically made this even worse because now it's so easy to generate slop and have it be recommended by Google's SEO. I used to be very good at phrasing a search query in the right way, or quoting the right words/phrases, or having it filter by sites. Those techniques no longer work.

So I have turned to ChatGPT for most of the queries I would have typically used Google for. Especially with the introduction of annotations. Now I can verify the source from where it determined the answer. It's a far better experience in most circumstances compared to Google.

I have also found ChatGPT to be much better than other LLMs at understanding nuance. There have been numerous occasions where I have pushed back against ChatGPT's answer and it has responded with something like "You would be correct if your input/criteria is X. But in this case, since your input/criteria is Y, this is the better solution for Z reasons".