Do LLM not also think when they encode the prompt? If Karpathy's explanation is accurate, longer prompts should also help even if they don't contain additional information, just by virtue of giving more time to think.
The time processing the longer prompt isn't being spent churning (i.e. "thinking") on the problem at hand, it's spend calculating attention matrices between all the tokens. The time spent on this is a function of the number of flops you have available.
So no, if you just fill up your context window to garbage, the LLM will not perform better at your task/question.
So no, if you just fill up your context window to garbage, the LLM will not perform better at your task/question.