Hacker News new | ask | show | jobs
by diggan 308 days ago
> All it takes is a simple reply of “you’re wrong.” to Claude/ChatGPT/etc. and it will start to crumble on itself and get into a loop that hallucinates over and over.

Yeah, it's seems to be a terrible approach to try to "correct" the context by adding clarifications or telling it what's wrong.

Instead, start from 0 with the same initial prompt you used, but improve it so the LLM gets it right in the first response. If it still gets it wrong, begin from 0 again. The context seems to be "poisoned" really quickly, if you're looking for accuracy in the responses. So better to begin from the beginning as soon as it veers off course.
2 comments
eru 308 days ago
You are suggesting a decent way to work around the limitations of the current iteration of this technology.

The grand-parent comment was pointing out that this limitation exists; not that it can't be worked around.

link
diggan 308 days ago
> The grand-parent comment was pointing out that this limitation exists

Sure, I agree with that, but I was replying to the comment my reply was made as a reply to, which seems to not use this workflow yet, which is why they're seeing "a loop that hallucinates over and over".

link
Breza 301 days ago
That's what I like about Deepseek. The reasoning output is so verbose that I often catch problems with my prompt before the final output is even generated. Then I do exactly what you suggest.
link