[ccozan]

yes, but why? Why is GPT so much better at programming than other tasks?

can it be that programming itself can be so easily predicted in a generative way, while others require more ingenuity and real world model to be solved?

In this case I would totally offload programming to a GPT /LLM AI, while my job is simply to specify largely the business case.

[impalallama]

I have to imagine its because so much of its training data is readily available programming docs, tutorials, and general Q&A that there is an amazing abundance of online. How many times have you just pasted an error into google and hoped someone else has asked the exact same question on stack overflow?

[petra]

True. Also there's a lot of commented open-source code out there.

[wprl]

As Chomsky points out, the AIs understand syntax not semantics. Code is all (or almost all) syntax whereas human language has both syntax and meaning. Meaning is not something ChatGPT understands.

[ccozan]

Thanks, that is the answer!

The meaning of a programming language is its syntax.

[lancesells]

Is it because programming is a more limited and specific language than the ones people speak? There's less room for double-meanings, slang, meaning, or even sentence structure.

[aero142]

It seems the improvement in the models is the size of the training set. Hence the "Large" in large language models. I am assuming that it is better at programming because there a is a ton of training data available on Github, etc. There isn't a similar set of data on solving a variety of physics problems.