[vlovich123]

> All in all, LLMs produced fewer equivalent mutants, mutants with higher fault detection potential, as well as higher coupling and semantic similarity with real faults.

The problem with PIT and Major is that they don’t do profile guided mutation testing [0] which in theory would raise the detectability rate without a meaningful cost increase. Other works explore the use of GANs [1] which would probably be cheaper and likely as effective but not as sexy as LLMs.

[0] https://arxiv.org/pdf/2102.11378

[1] https://ar5iv.labs.arxiv.org/html/2303.07546

[jungsteven]

Thanks for sharing the papers! I remember reading the first one from Google and can’t wait to dive into the new one. Appreciate the insights!