[delusional]

Did you continue reading? His argument is exactly that, like all the previous model checkers, LLM's are going to give us some bugs for a while. Then that is going to stop.

His argument is not that they aren't going to find any bugs, but rather that at some point those bugs will be fixed. At which point we will continue on as usual.

[ang_cire]

How would they stop? Are you insinuating they're going to have reviewed all code at some point, and that new code for them to review will just cease to exist? Or that they'll just decide to stop finding bugs they're finding now in new code when they review it? All the previous model checkers didn't stop giving us bugs to fix, which is why his premise is wrong; every big company is still running SonarQube because it still gives you bug findings. So will AI.

If the argument were instead that it will cease to find new classes of vulnerabilities and bugs, that may very well be true, because that is a question of the limitations of programming and at a lower level computer architecture, but that's not the argument the author made.

[erikerikson]

The author is talking about released bugs. The tools will then continue finding the bugs they find in new code, keeping those from being released. At least that was my reading of the article.

[refulgentis]

I did, his argument is that we've already discovered ~50% of all bugs discoverable by LLMs.

I'm treading lightly after you said "did you read it" to OP, I do believe we both understand that argument isn't nearly air-tight. (i.e. it implies either humans get so good at code that bug-introduction-rate falls percipitously, or, LLMs are so awesome they write all of our code bug-free. Neither of which jives with the thesis, that LLM code review is a nothingburger long term)

The best steelman we could say is "he meant 50% of all existing bugs in all currently existing code", which is still incompatible with a time-bound on their usefulness, unless we expect the rate of new code to fall percipitously.

The steelman I'm using, is they're speaking both loosely and strongly and intend us to understand these are strong opinions, held loosely, and they care for us enough to share.

[simianwords]

> At which point we will continue on as usual.

This part is the load bearing claim. Why would you continue on as usual? I'm using LLM's everyday on code reviews and they still catch bugs.

[layer8]

The question is whether the balance of white-hat and black-hat LLM bug findings gives us a different enough result from the previous balance of white-hat and black-hat human bug findings to constitute a major disruption.

[simianwords]

I don't understand your point. How do you make your future code resistant to bugs without LLMs?

[layer8]

That wasn’t the question. The question is how LLM code review is a huge disruptor. What is it disrupting? At best, with regard to code review LLMs just solve the problems that LLMs create.