[SeanAppleby]

GPT-2 doesn't do a _great_ job at this, but it does to some extent seem to have learned some relationships in structures of text in different programming languages.

For example while toying around with it, I gave it a javascript function that referenced an undefined callback, and GPT-2 gave me back a function, with almost correct syntax and mostly random logic, but with the name and call signature of the callback.

[6gvONxR4sf7o]

These sorts of models do well at generating text, whether it's code or language. That's what you're talking about. They can perform the "execution" to some degree too for natural language (e.g. the squad tasks). But Q&A and text generation are very different tasks. I haven't seen anyone apply a transformer to source code based Q&A tasks.