[chowells]

Just take a look at the generated code and explanations. A surprising amount of it is subtly but fundamentally wrong because gpt is just a regurgitation engine. The issues may look superficial, but when you start looking at why they happen, you realize the truth. The ml tools are usually great at writing boilerplate that's the same every time. The instant you do anything else, they fall over. They're statistical autocomplete, not any kind of important change to the process of programming.

I don't see any reason to believe the current approaches can extend to something that actually changes programming. They're not based on understanding code, they're based on generating text that matches what they would expect to see given the context. They have no model of what code means, so they can't model why sometimes code is subtly different if there are no local contextual cues. And when there are your prompt would need to reproduce those contextual cues for it to key off of. In other words, you as the programmer still are directing the generation of the code. You're just doing it via an undocumented and somewhat unpredictable autocomplete.

This doesn't remove the need to have someone who knows what they're doing in the loop. Best case is that it reduces the amount of time you spend typing by a little bit. As long as your job is to know what you're doing rather than to generate text, the current systems are no threat to it.

[jmfldn]

This was pretty much my take too until I played with GPT 3. I suspect that you're still basically right, but the code it was writing, whilst a bit quirky and sometimes full of errors, showed a simulacra of creativity. I use this term as I know it's an illusion, it is what you suggest, but it's amazing to me that this trick can be pulled off. I got the program to write some fairly esoteric functional programming code. I then pasted some of my own code and got a convincing "explanation". If nothing else, if GPT 3 can simulate understanding in some narrow cases through what is essentially a giant search engine trained on a gazillion data points, then it's a good trick.

It's quite possible that this avenue doesn't scale to anything more broadly useful. We shouldn't mistake solving 20% of a problem to being on the right path. Maybe this remains as auto-complete on steroids and it's a dead end. I was honestly just surprised by GPT 3's apparent abilities, smoke and mirrors though they may be!

[haolez]

Sounds good, but... do we have models in our heads? Or are our minds simply huge and complicated neural networks?

[chowells]

Well, we absolutely have models in our heads. That's how we can understand what programs do.

It's possible our minds are also huge and complicated neutral networks, though I suspect that description is incomplete at best.

But the point is that current tools are trained on text generation. Something that would change programming would have to train on the meaning of programs. It's a rather different task, as it's no longer statistical. Doing it properly requires metacognition as well, to avoid falling into the trivial inconsistencies in most programming languages. And connecting that with real-world tasks that it hasn't seen before would require an understanding of the real world.

I'd call something with all of those capabilities AGI. I honestly don't think any system short of that will ever be more than an autocomplete, because it can only ever fit things together based on some statistics.