I expected more from the study. Maybe actual examples on simple realistic prompts on the same issue having polar opposite answers, bases only on phrasing or something akin
From my experience, for any even a tiny bit ambiguous topic where X = !Y, when I ask ChatGPT “Is X true?” it usually responds yes and follows up with some supporting arguments for X.
If I ask “Is Y true?” it tells me Y is indeed true and explains some reasoning.
Therefore I try to always inquire in the form of “Which one is true, X or Y?” to avoid the yes bias.
Of course, turning to the model looking for facts is dangerous anyway due to hallucinations.
No, the correct way is to have it reason from first principles:
1. "Think about what are the underlying principles for evaluating the truthiness of statements like 'X' - list them out, explain why you chose each one, what tradeoffs you made, why you believe it's the right tradeoff in this case"
2. Start a new conversation and make the system prompt be that set of principles
3. In the user prompt, ask it to decompose X into a weighted formula for those principles and give a sub-score for each principle.
4. Finally, based on the weighted sum, ask it to determine if X is true or not true, and ask it to provide a confidence score between 0 and 1 for its response
If I ask “Is Y true?” it tells me Y is indeed true and explains some reasoning.
Therefore I try to always inquire in the form of “Which one is true, X or Y?” to avoid the yes bias.
Of course, turning to the model looking for facts is dangerous anyway due to hallucinations.