Hacker News new | ask | show | jobs
by CuriouslyC 2 days ago
The pretraining data isn't getting muddled, the instruction just gets in the lossy middle, and there's less for attention to grab on to. If you repeated the prompt with a hook every turn, it'd stay quite sharp.