[wgd]

Yes, those ones would be at least a somewhat-plausible simulation of a real scenario people care about: a once-clean codebase that was allowed to become messy by a succession of insufficiently-careful vibeslop PRs.

I'm not a huge fan of their methodology for the AI-degraded cases either (ideally one would set up the mirror pairs by taking some real repositories and rewinding history a month or so and then having a succession of independent agents reimplement each bit of feature work and bugfixes over that period of time), but it's at least a coarse approximation whereas I just don't trust the cleanup methodology to resemble anything real in the first place.

[geraltofrivia]

Our initial suggestion was to do something along the lines of your proposal. But we found that when we ask an agent to implement features for `t-n`^th PR, even when we are overly specific makes the code rather divergent, to the point that sometimes the `t-n+1`th task description doesn't really make a lot of sense.

In fact, there are some papers (that we cited) which create a set of tasks doing exactly this, and it is non-trivial [1].

[1] https://arxiv.org/html/2603.24755