[koakuma-chan]

Obviously not going to reply to everybody, but many people here got confused thinking that the LLM two million input tokens is analogous to their brain long-term memory. This is not true. The LLM two million input tokens is more like your brain "working memory," and your brain long-term memory is more like the LLM training data.

I do agree with you that LLMs get "confused," failing to follow constraints. This is also my experience. But, the reason for this phenomena is the lack of emphasis on your constraints.

For example, when working with stable diffusion, you can manipulate the weight of parts of your prompt. Say, you wanted to generate an image, and you really wanted there to be a dog, you could prompt: "a clear sky under the moonlight, (dog:1.5)," and this case the model would give the "dog" part would be 1.5x more important to the model then the rest of the prompt. Not sure why there is no such feature for LLMs (it could be, just that I'm not aware).

I looked at your prompt history with the German article, and I can see that the reason it fails is that you prompt incorrectly. When you want to give certain information or context to the LLM, say, your codebase, or some documentation, or some article, you gotta put it first in your prompt, and at the very bottom you should put your instructions. This apparently makes it easier for the LLM to parse your request.

Also, generally, LLMs will not give you a response longer than a few thousand tokens, so what you should do is: ask it to translate it section by section, and keep asking "Translate the next section," until it translates all of them. I was able to translate your article this way using Gemini, not sure how accurate it is though.

[ben_w]

> I looked at your prompt history with the German article, and I can see that the reason it fails is that you prompt incorrectly. When you want to give certain information or context to the LLM, say, your codebase, or some documentation, or some article, you gotta put it first in your prompt, and at the very bottom you should put your instructions. This apparently makes it easier for the LLM to parse your request.

Good to know — I somehow failed to be aware of this before now despite playing with these models since the days of AI Dungeon (for open source models) and text-davinci-003 (from OpenAI).

> I was able to translate your article this way using Gemini, not sure how accurate it is though.

The second part is important — checking the translation was how I knew the it was making things up in the "translated" timeline.