[entrope]

The regressions got fixed before a release. Do you count your internal integration branch as "shipped"?

0.02 defects per KLOC is a low density, even considering that it was "just" a port to a new language and before giving credit for the existing bugs it identified and fixed. That defect density will probably only generalize to other projects that have similarly extensive regression test suites, but it's a proof of what LLMs can do.

[SpicyLemonZest]

The source article says that "Large code migrations are a particularly effective use case for these advanced models", and the specific advantages suggest to me that the results won't generalize at all. I can't think of anything other than a large code migration that would allow for thousands of independent workstreams against a sufficiently detailed spec to stop agents from going completely off track.