[mkleczek]

I am very sceptical of LLM (or any AI) code generation usefulness and it does not really have anything to do with AI itself.

In the past I've been involved in several projects deeply using MDA (Model Driven Architecture) techniques which used various code generation methods to develop software. One of the main obstacles was the problem of maintaining the generated code.

IOW: how should we treat generated code?

If we treat it in the same way as code produced by humans (ie. we maintain it) then the maintenance cost grows (super-linearly) with the amount of code we generate. To make matters worse for LLM: since the code it generates is buggy it means we have more buggy code to maintain. Code review is not the answer because code review power in finding bugs is very weak.

This is unlike compilers (that also generate code) because we don't maintain code generated by compilers - we regenerate it anytime we need.

The fundamental issue is: for a given set of requirements the goal is to produce less code, not more. _Any_ code generation (however smart it might be) goes against this goal.

EDIT: typos

[mvdtnz]

You should NEVER modify generated code. All of our generated code is pretended with a big comment that says "GENERATED CODE DO NOT MODIFY. This code could be regenerated at any time and any changes will be lost."

If you need to change behaviour of generated code you need to change your generator to provide the right hooks.

Obviously none of this applies to "AI" generated code because the "AI" generator is not deterministic and will hallucinate different bugs from run to run. You must treat "AI" generated code as if it was written by the dumbest person you've ever worked with.

[fragmede]

The reason you don't modify generated code is it gets clobbered upon regeneration. The reason it's okay to modify LLM-generated code is that it gets fed that back into the LLM for subsequent modification.

[mkleczek]

That's exactly my point :)

[smokel]

I agree. Adding unit tests without a good reason comes at a cost.

Refactoring is harder, especially if it's not clear why a test is in place. I've seen many developers disable tests simply because they could not understand how, or why, to fix them.

I'm hopeful that LLMs can provide guidance in removing useless tests or simplifying things. In an ideal future they may even help in formulating requirements or design documentation.

[mkleczek]

> I'm hopeful that LLMs can provide guidance in removing useless tests or simplifying things. In an ideal future they may even help in formulating requirements or design documentation

I am very sceptical here as well. The biggest problem with formulating requirements or design documentation is translation from informal to formal language. In other words... writing programs.

LLMs are good at generating content that doesn't provide useful information (ie. have low information content). Their usefulness right now is caused by the fact that people are used to reading lot of text and distill information from it (ie. all the useless e-mails formulated in corporate language, all multi-page requirement documents formulated in human readable form). The job of a software engineer is to extract information from low information content text and write it down in a formal language.

In this context:

What I expect in the long run is that people will start to value high information content and concise text. And obviously - it cannot be generated by any LLM, because LLM cannot provide any information by itself. There is really no point in: provide short high information content text (ie. prompt) to LLM -> receive long low information content text from LLM -> extract information from long text.