[ori_b]

> Theorem proving tools would work if the students wrote their answers in a format that the tool would work in.

Theorem proving tools would work if you were writing formal proofs. At the level he's talking about, students are not writing formal proofs -- they're writing explanations.

[gbhn]

Right. It seems to me that the despair is that it is very hard, from knowledge like "student checked box B in this test, which was correct" to deduce "student actually understands the concept".

Perhaps a partial solution to this problem can be found in... wait for it... programming! That is, it is really hard to write a program to find general answers to problems unless you understand the basic idea. This definitely doesn't work for everything, but I wonder how big the domain is where this is a really good way to mechanically assess understanding, and whether the language that'd be required would itself become a bigger component of the measure than what you're trying to assess -- mathematical reasoning.