Hacker News new | ask | show | jobs
by CharlesW 26 days ago
> The trick is to have a fresh instance doing the reviewing, not the one that did the work.

In my experience that's not neccessary (some people even claim that you must use models from different vendors), and it's expensive since a fresh instance needs to rebuild all the context that's needed in order to properly and thoroughly review. LLMs have no problem throwing "them 5 minutes ago" under the bus when asked to review something "skeptically" and "with fresh eyes".

2 comments

Doing it in the same session does save a ton of tokens but I find it's too biased towards its own implementation even if you tell it to use "fresh eyes" or to "act like a code reviewer in a bad mood." Including those strings in your prompt does show some improvement but not nearly as much as making it think from first principles in a fresh instance.
Is there research about this?

That sounds like it would make productive AI usage much easier, but it also sounds very brittle