[chirau]

then perhaps you should tell it to lie to you, no?

Prepend that to your prompt perhaps. Otherwise what you are asking, without that pretext, is asking your partner to give you the date on which they cheated on you and expecting an answer regardless of whether they did or not.

[drusepth]

If I asked my partner to provide an argument for why earth is flat, she would do it. She doesn't think (or have to think) the earth is flat to make an argument.

I'd expect an AI trained on human conversation to act the same and I'd be frustrated if it declined to do so, the same way I'd be frustrated if a friend also declined to do so.

[chirau]

It does that too. As I stated on the parent comment, just add 'hypothetically' to the prompt. It also categorically dismisses it after all the spiel.

[drusepth]

Yeah, the humans I'm referring to don't need the hypothetical prefix, nor do they go out of their way to categorically dismiss everything they've said. That's the difference.

But it's not a hill I want to die on, especially when there are other LLMs I can just switch to that act more how I'd hope/expect.