[dgb23]

I don’t think that’s the point of showcasing these issues.

The specific point is that you cannot prompt your way to reliable software (AKA vibe coding). Just as you cannot reach the same goal by glueing together stackoverflow snippets without understanding them.

[Dig1t]

I understand that, but the interesting bit is to compare how it performs relative to the average human coder. We can point out specific flaws for eternity, but if it makes 1% fewer mistakes or allows humans to code faster without increasing the number average number of mistakes, then I'd say that that's still providing value. I feel like just enumerating different mistakes that it's made is sort of biased against it because it leaves out a comparison to the alternative.

Sort of like showing off self-driving car crashes. You can spend all day listing the crashes and showing people how it has problems, but if it's statistically safer than the average driver it would save thousands of lives per year to deploy it anyway even if it's not perfect.