Hacker News new | ask | show | jobs
by sacred_numbers 1174 days ago
The Reflexion paper (https://arxiv.org/abs/2303.11366) that came out recently shows how this kind of mistake might be overcome. Asking the model to think about the answer after it's generated a first draft greatly improves accuracy. Also, prompt engineering such as copying the generated code, pasting it in a new chat and saying "There's a bug in this code, please find it" can go a long way. There is so much low hanging fruit in harnessing the power of these models that is just being ignored because some even lower hanging fruit (RLHF, system messages, context window size, plugins, etc) is being released seemingly every few days.
2 comments
yawnxyz 1174 days ago
That’s really interesting! It’s like treating this thing like a kid and telling them to really think about the answer and show the work. Weird!
link
Vespasian 1174 days ago
I recall reading that's partly because it doesn't have an inner monologue.

It only "knows" what it writes down and if you force it to print the intermediate step it can more accurately get to the final answer.

link
wolfi008 1174 days ago
> Asking the model to think about

These models do not "think". This is a fundamental misunderstanding of how they work. It's not AI. It's not even language. It's just text inference.

link
pmx 1174 days ago
If you ask the model to "think" about something, and then it simulates that action and outputs what the result of that might be, does it matter if it's really thinking or not? Especially if the output is what we wanted originally?

I would suggest that a person saying "ask the model to think about" in this context in no way implies that that person is confused about the nature of the model, it is simply a convenient piece of language that helps us to achieve the desired result.

link
wolfi008 1174 days ago
> does it matter if it's really thinking or not?

Yes, it does. Assuming GPT "thinks" is shaping public perception and that's already gone much too far.

link
silveraxe93 1174 days ago
He did not say "make the model think about..." or implied the model is thinking. He simply and _correctly_ pointed that if you _ask_ the model to think it improves the answer.

It looks you just pattern-matched on the word _think_ and replied with a pre-made opinion about how AIs can't think. Ironic...

link
themodelplumber 1174 days ago
Maybe you are asking it to think and it's like, "this is 99% likely just code analysis requested"

So it's still fair to say you can ask it to think

link