[ariofrio]

This has certainly been my experience. It's pretty hard to get ChatGPT to tell me I'm fundamentally wrong about something. But I see the problem as similar to getting feedback from people, since most people are hesitant to outright disagree with you. So you have to phrase things in ways that encourage the person to be more forthright, and this works to an extent with LLMs as well.

[sitkack]

If you want constructive feedback and be told you are wrong then you need to craft a system prompt for it. They have been trained to be agreeable and tell you your ideas are great.

[A_D_E_P_T]

If you ask for an assessment of something, it's often as easy as adding "be critical" to your prompt. e.g.:

"What do you think of this short essay?"

"What do you think of this short essay? Be critical."

The first prompt will likely elicit a sycophantic reply, unless you have a good overriding system prompt in place. You'll always get much better feedback with the second.

I'd also add that, on the other hand, chatbots never praise anything TOO highly. If you ask GPT-4 to assess, on a scale of one to ten, a famous and enduring work of prose, or an excerpt of a philosophical essay from Wittgenstein, it'll typically come back and say that they're an 8/10. Rarely 9/10. Never 10/10, no matter what you submit.

[anigbrowl]

Getting LLMs to argue or to ask for more context rather than always offering an answer are among the most difficult interactions to elicit. There's a big gap between helpful and obsequious, and unfortunately society often selects for teh latter.