[lacunary]

The develop-test-refine feedback loop for this kind of attack is so long (or expensive) that it seems likely to limit its real world use. Poison training data, wait months? a year? for the model to come out, see how well it worked, refine... or do you see a faster way to iterate?

[OtherShrezzing]

Continual learning is the next major architectural milestone for the frontier labs. That’d reduce the iteration loop to days instead of years.

If your attacker assumes that all or most software will be generated from language models, the time penalty is worth paying.