[1659447091]

If a used car salesman says a specific car is in great condition and fits all of your needs, you don't verify their claim by asking what the downsides are. You take the car to a trusted mechanic and have them look it over before finalizing the contract.

[lunarcave]

Yeah, but the used car salesman's explicitly disincentivised to tell you the downside. An LLM is agnostic to the outcome. It's only incentive is to produce an output.

A correct analogy here would be your distant uncle who knows about cars. He doesn't really care whether you buy it or not. He only cares he imparts some information which is helpful.

[1659447091]

The point was really when you want to verify something seek a 2nd opinion.

An LLM does not care if it's information that it is imparting with is helpful or not, unlike the uncle. Maybe it will be maybe it won't be. Either way, seek a 2nd opinion.

So maybe doctor giving a diagnoses and telling you about the recommended procedure. You can ask the downsides and get them. But you don't go with the first thing the doc says, most people seek a 2nd or third opinion from other sources to verify if the information first given is right for you or if there may be better options etc