[data-ottawa]

if you shrink the context window on most models you'll get this type of behaviour. If you go too small you end up with basically gibberish even on modern models like Gemini 2.5.

Mercury has a 32k context window according to the paper, which could be why it does that.

[jdiff]

I've run into this even with the modern million context length that 2.5 Pro offers, it kept trying one of a handful of failed approaches, realizing its failure, and looping without ending its train of thought until I yanked the tokens out of its mouth.

Even though it has gotten drastically better and rarer, I think this is going to be one of the failure modes that's just fundamental to the technology.