Claude sonnet 4 (this time last year) did do this. It once made simulation if a test script passing. Literally a script that just echoed test names and then said pass.
Happened to me, 3 days ago - deleted some tests and flipped assertions after outlining that it wasn't to change any assertions.
Our team was doing a similar task to move between test frameworks, and I had to do a git diff of hundreds of thousands of lines to try and work out where a test had disappeared to.
> Change happens fast, a year old model is pretty outdated.
What change? That you should not fake the results of a test because that defeats the whole purpose of a test has been known before there were computers.