[tanilama]

I mean it is cool.

But there is the thing, the natural description of a function is not always this unambiguous.

When you are telling a function to 'compute XYZ', what you are actually doing is 'check whether X.a exists, if so execute branch 1), else branch 2)'.

If the logic gets really complicated, then describing it accurately in human language isn't necessarily faster than doing it in code directly. Otherwise, we don't need invent programming languages like at all, we can just write compilers to interpret and execute human languages.

And I am interested, as whether the model itself is conditioned on the type constraint of class. It is neat that they pick Python in this case. But if it is Java or other static typed language, would this system condition its generation not only the natural text, but also the resulted type system? My bet, per my understanding of the language modeling approach they use is, they are not doing this, due to very high complexity and cost of the training, and domain adaptation.

Overall, this again is an interesting demo. But I think for code generation based on human language to be useful, we are really in a scenario, that you need to go 99% accurate for it to be remotely practical.

[nerdponx]

This might be more useful for a task like "read files off a list, and download them in parallel, with no more than 20 concurrent downloads." That particular task might be a one-liner in some programming languages, but there are a lot of programs like that which need significant bookkeeping and/or boilerplate even though their plain-language description of intended behavior is not complicated.

Or implementing a sophisticated protocol that has a formal specification. If you can express the correct behavior in some kind of pithy pseudocode, a tool like this could "compile" that to code in various programming languages. Like a super-powered version of SWIG.

[MiroF]

I agree that code generation of complex functions is hard.

But I think the example given of unit testing - ie. natural language description of specific behavior of function -> code is extremely useful.

[tanilama]

Unit testing is a good use case.

But that would require the condition on the type system, meaning the code-gen needs to understand the object's interface, which while not impossible in current techniques, but hard enough due computation complexity.

Again I don't dispute this tool being interesting. But claims it to be ground breaking or game changing is simply not right.

Majority of programmers time, is not typing down the code. It is to look at the comment/description, think about it, edit some code, then rethink then edit again.

This tool has potential to solve some typing time, but it still not going to things fundamentally.