Hacker News new | ask | show | jobs
by saaaaaam 146 days ago
> Model performance has also been shown to be better if you lead with the question. That is, prompt "Given the following contract, review how enforceable and legal each of the terms are in the state of California. <contract>", not "<contract> How enforceable...".

Confused here. You attach the contract. So it’s not a case of leading with the question. The contract is presented in the chat, you ask the question.
1 comments
avidiax 146 days ago
LLMs are necessarily linear. If you paste the contract first, the attention mechanism of the model can still process the contract, but only generically. It pays attention to the key points of the contract. If you ask the question first, the attention part of the model is already primed. It will now read the contract paying more attention to the parts that are relevant to the question.

If I ask you to read Moby Dick and then ask you to critique the author's use of weather as a setting, that's a bit more difficult than if I ask you to to critique that aspect before asking you to read the book.

link
saaaaaam 145 days ago
No, but I mean that in Claude you don’t put the contract linearly into the chat - in other words you can’t position it before or after the prompt, you attach it at the top of the chat. Are you saying you would prompt saying “please examine the contract I will provide in the next message, here is what I want you to do <instruction>”
link
avidiax 145 days ago
The LLM developers already know this trick, so I expect that if you attach documents, they are processed after your prompt.

There is a further trick that is probably already integrated: simply giving the same input twice greatly improves model performance.

link
saaaaaam 145 days ago
Gotcha, thanks for explaining. It’s interesting because there are times I say “look at his document and do this” but forget to attach the doc. I’ve always had the sense Claude is better “prepped” when it anticipates he document coming. Sometimes I’ve said “in the next message I’m going to give you a document, here is what I want you to do, if there’s anything you are unclear in before we begin ask me questions”. This seems to bring better results, but I’ve not done any sort of robust test.
link