[richardw]

I think so. You ask that question because you’re interrogating the position, not because 1000 humans have asked that question in similar situations.

You and I know there’s a truth and we’d like to find it. The GPT is just happy (I.e. rewarded) to produce frequently used tokens.

[mewpmewp2]

And I'm just happy to perform actions that will make me survive and reproduce?

[richardw]

Most likely, unless you meditate a lot. Sometimes you'll take a bullet to save other people. Sometimes you'll drink yourself into a state that doesn't help you survive or reproduce. Or you'll write on a forum anonymously that doesn't help with survival or reproduction because it's enjoyable, makes you think, or you're addicted. Who knows :)

[mewpmewp2]

You are even better at analyzing me than GPT-4.

[parpfish]

but maybe that feeling of 'looking for truth' is just what happens when you're doing pattern matching on the text embeddings?

[richardw]

I feel it’s a bit more, given we think about it after the sentence is complete. But it raises some interesting questions about what an agent would do if it had an instruction to keep trying until it got a reliable answer. Maybe an argument generating agent and a critic agent.

Worth a shot :)