In my view the difference comes down to accountability. In all of the heavily-regulated environments I’ve worked in, humans are responsible for compliance in systems.
If an LLM generates vulnerable code that evades detection and makes it into production, and that vulnerability is exploited, who is responsible? Presumably, the humans who were tasked with reviewing and approving the code.
As the famous quote states, it is easier to write code than to read it. An LLM would be great for improving developer tooling to generate boilerplate code, etc., but I see tremendous risk for any firms attempting to let an LLM design and build any substantial non-trivial pieces of system code. And the effort required to review the code is at least on the same order of magnitude as it would be to write the code (beyond the trivial “make me an HTTP controller or HTML form for these operations” and the like)
If an LLM generates vulnerable code that evades detection and makes it into production, and that vulnerability is exploited, who is responsible? Presumably, the humans who were tasked with reviewing and approving the code.
As the famous quote states, it is easier to write code than to read it. An LLM would be great for improving developer tooling to generate boilerplate code, etc., but I see tremendous risk for any firms attempting to let an LLM design and build any substantial non-trivial pieces of system code. And the effort required to review the code is at least on the same order of magnitude as it would be to write the code (beyond the trivial “make me an HTTP controller or HTML form for these operations” and the like)