|
|
|
|
|
|
by Der_Einzige
780 days ago
|
|
|
I have tried a lot of RAG and can tell you that no LLM, including Gemini 1.5 with it's 1.5 million context, will be anywhere near as good at longer context lengths as in shorter context lengths. Appending huge numbers of tokens to the prompt often leads to the system prompt or user instructions being ignored, and since API based LLM authors are terrified of jailbreaks, they won't give you the ability to "emphasize" or "upweight" tokens (despite this being perfectly possible) since you can easily upweight a token to overwhelm the DPO alignment lobotomization that most models go through - so no easy fix for this coming from OpenAI/Anthropic et al |
|
|