[bugglebeetle]

Of course, but ease of detection can vary relative to the complexity of the code being returned. GPT-4, correctly prompted, can produce some pretty complicated stuff. But it also hallucinates in ways that are more subtle than one might think. The example I’m thinking of, it created an unused variable in a set of fairly complex ML training set up scripts that I mostly caught because I was familiar with all the proper inputs. But the unused variable was quite plausible if you were not familiar, new to the domain etc.

[gowld]

Compilers automatically detect unused variables. Unused variables are the last of your problems. You should be far, far more worried about all the misused variables.