[johndough]

> Code seems at its surface a much simpler language

When using GitHub Copilot, I often write a brief comment first and most of the time, it is able to complete my code faster than if I had written it myself. For my workflow, a good code model must therefore also be able to understand natural text well.

Although I am not sure to which degree the ability to understand natural text and the ability to generate natural text are related. Perhaps a bit of text generation capabilities can be traded off against faster execution and fewer parameters.

[GeneralMayhem]

Understanding should be much easier, for the same reason humans (e.g. children, foreign-language learners) can always understand more than they can say: human language is fairly low-entropy, so if there's a word you don't understand, you can pick up most of the meaning from context. On the other hand, producing natural-sounding language requires knowing every single word you're going to use.