[fergal_reid]

Huge respect for Norvig, but I think this is a shallow analysis.

For example, I just took Norvig's 'backspacer alpha' function and asked ChatGPT about it. It gave me an ok English language description. It names the variables more descriptively on command.

I'm sure it'll hallucinate and make errors, but I think we're all still learning about how to get the capabilities we want out of these models. I wouldn't rush to judgement about what they can and can't do based on what they did; shallow analysis can mislead both optimistically and pessimistically at the moment!

[neontomo]

You are talking about a completely different language model though, so your argument doesn't really hold. The point he makes is that AlphaCode does not generate meaningful explanations along with its code (and that's it's perhaps necessary to build trust in a system), not that there aren't any language models that can do it. AlphaCode !== ChatGPT. I encourage you to write a better analysis!

[fergal_reid]

Well, the stall he sets out is about 'generative models' generally:

"In the future, what role will these generative models play in assisting a programmer or mathematician?"

I think the fact that we've other generative models that eg. can write descriptive variable names is a fair rebuttal to his criticism of alphacodes poor variable names.

Deepmind fine tuned it on competition code, not on well documented production code, so I think it's shallow to get hung up on it's lack of good variable names (for example).