[jmfldn]

This was pretty much my take too until I played with GPT 3. I suspect that you're still basically right, but the code it was writing, whilst a bit quirky and sometimes full of errors, showed a simulacra of creativity. I use this term as I know it's an illusion, it is what you suggest, but it's amazing to me that this trick can be pulled off. I got the program to write some fairly esoteric functional programming code. I then pasted some of my own code and got a convincing "explanation". If nothing else, if GPT 3 can simulate understanding in some narrow cases through what is essentially a giant search engine trained on a gazillion data points, then it's a good trick.

It's quite possible that this avenue doesn't scale to anything more broadly useful. We shouldn't mistake solving 20% of a problem to being on the right path. Maybe this remains as auto-complete on steroids and it's a dead end. I was honestly just surprised by GPT 3's apparent abilities, smoke and mirrors though they may be!