[voxgen]

There is a third case where the other party doesn't realize that the asker lacks the relevant experience to discern good LLM answers from bad answers for that topic.

Same solution as case one though - don't be afraid to say "Claude said X but that doesn't sound right".

[14u2c]

Depending on the question that possibility can be quite rare these days. If you ask “How does x work in this codebase?”, it will read a bunch of files and give you a very likely to be accurate answer. If your using a platform without that context and ask it something more abstract, well, your mileage will vary.

[mlinhares]

Here's the problem, countering a lie or hallucination takes much more energy than asking Claude something and saying its true. Its the same as trying to fight misinformation on the internet, the amount of energy you have to spend to prove someone is lying or fabricating data is very high.

And having to do this on the corporate environment saps the energy and time of people that could be doing something productive by wasting their time answering a clueless person that asked an LLM about something they don't understand, got the answer they wanted (but that isn't real), and now are asking multiple people to prove it can't be done.

Here's an example, a PM decided they wanted to build a metrics framework, to track team success, with high level metrics. They asked claude to build such high level metrics (out of nowhere, these metrics don't exist), it happily produced hallucinated code that said it was collecting the metrics and the PR opened a pull request. Now we have to go there, review, find out is all bullshit and explain to the person that what they're trying to build doesn't exist.

So now we have to fight misinformation even on the clock.