[gbhn]

Right. It seems to me that the despair is that it is very hard, from knowledge like "student checked box B in this test, which was correct" to deduce "student actually understands the concept".

Perhaps a partial solution to this problem can be found in... wait for it... programming! That is, it is really hard to write a program to find general answers to problems unless you understand the basic idea. This definitely doesn't work for everything, but I wonder how big the domain is where this is a really good way to mechanically assess understanding, and whether the language that'd be required would itself become a bigger component of the measure than what you're trying to assess -- mathematical reasoning.