[jorl17]

Yep, I did. It didn’t have any effect. Randomizing the poems has stopped having a significant effect in the last year or so on the analysis as a whole.

It still somewhat affects the way the AI looks at the poems, especially if it has to make lists of the “best”, but it is not a very pronounced effect nowadays. Maybe it gives some preference for earlier and latter poems but not a lot.

Back when I started doing it, these holes in the context were super obvious, just like you described. It would mixup entire periods and also hallucinate or forget about specific sections. That’s precisely why I started randomizing them as part of my experiments.

Nowadays the frontier models are much better and don’t really mix anything up.