[dgfitz]

Wait, what?

Isn't that the whole point, to ask it specific tidbits of information? Are we to ask it large, generic pontifications and claim success when we get large, generic pontifications back?

The narrative around these things changes weekly.

[simonw]

ChatGPT is exceptionally good at using search now, but that's new this year, as of o3 and then GPT-5. I didn't trust GPT-4o and earlier to use the search tool well enough to be useful.

You can see if it's used search in the interface, which helps evaluate how likely it is to get the right answer.

[rollcat]

The problem is, I ask it a basic question, it confidently feeds me bullshit, I correct it twice, and only then it does an actual search.

[simonw]

I use GPT-5 thinking and say "use search" if I think there's any chance it will decide not to.

[rollcat]

This is what I have in my custom instructions:

    Stay brief. Do not use emoji.
    Check primary sources, avoid speculation.
    Do not suggest next steps.

Do I have to repeat this every time I suspect the answer will be incorrect?

[dpkirchner]

I use it as a tool that understands natural language and the context of the environments in work in well enough to get by, while guiding it to use search or just facts I know if I want more one-shot accuracy. Just like I would if I were communicating with a newbie who has their own preconceived notions.

[wredcoll]

I mean, like most tools they work when they work and don't when they fail. Sometimes I can use an llm to find a specific datum and sometimes I use google and sometimes I use bing.

You might think of it as a cache, worth checking first for speed reasons.

The big downside is not that they sometimes fail, its that they give zero indication when they do.