[benhoff]

I've often thought it would be cool to use AST's and perhaps code embeddings generated from machine learning as a tool to help students improve.

If you've ever taught a course with intro level python, it quickly becomes apparent how repeatable the mistakes are, or where you didn't spend enough time. As a student, this is frustrating because the correction comes too late, it's why having someone knowledgeable over your shoulder can speed up your learning.

The challenge that I believe ASTs present is that they only parse compliant code. So if someone makes a syntax error, it becomes a whole new ball game. I'd glanced at tree sitter to see if this could fix some of these issues, but I think it's a more fundamental problem than that.

[tusharsadhwani]

tree sitter can definitely help with this problem, but so can regular AST parsers, the idea is the same: just add code or grammar that will parse the "invalid" grammar, mark it as invalid, and continue parsing valid code as soon as possible.

Existing code editors like VSCode do exactly this for better syntax highlighting of incomplete code.

[hsbauauvhabzb]

Wouldn’t that be impossible? The structure of python is finite, and invalid deviations are infinite. Sure any language AST compiler could be more helpful, but they can’t take trash and turn it to gold.

[TuringTest]

In the context of programming lessons, there is a known correct program. Wouldn't it be possible to calculate the distance between what has been typed and how the correct finished program should be, to guide students into correcting the non-parsing parts of their code?