That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.
Because the behaviour of software changes over the time of development and that's how many bugs happen in the first place.
Especially if you use AI, let's say you have it implement a feature and then change your mind. In my experience AI makes as many if not more bugs than a human.
It’s not necessarily symmetrical, and in fact would be very surprising if it was. It’s a probabilistic algorithm on both sides, so the energy use to find any working program vs all bugs in a working program are fundamentally different search spaces. Not to mention the false positive rate and the human verification effort. Then even the idea of incremental security checks is potentially flawed since many security issues are non-local (ie not localized to a single module).
I think I’m on this side. I find it exceedingly unlikely that we just start producing “perfect” software all the time for everything, and at the same time start generating an order of magnitude MORE software.
> That supposes that LLMs can write secure software.
I think we're at the point that the best LLMs can indeed write software that's far more secure than your average programmer. Partly because the average is so terrible.