[polyrand]

There is too much focus on students cheating with AI and not enough on the other side of the equation: teachers.

I've seen assignments that were clearly graded by ChatGPT. The signs are obvious: suggestions that are unrelated to the topic or corrections for points the student actually included. But of course, you can't 100% prove it. It's creating a strange feedback loop: students use an LLM to write the essay, and teachers use an LLM to grade it. It ends up being just one LLM talking to another, with no human intelligence in the middle.

However, we can't just blame the teachers. This requires a systemic rethink, not just personal responsibility. Evaluating students based on this new technology requires time, probably much more time than teachers currently have. If we want teachers to move away from shortcuts and adapt to a new paradigm of grading, that effort needs to be compensated. Otherwise, teachers will inevitably use the same tools as the students to cope with the workload.

Education seemed slow to adapt to the internet and mobile phones, usually treating them as threats rather than tools. Given the current incentive structure and the lack of understanding of how LLMs work, I'm not optimistic this will be solved anytime soon.

I guess the advantage will be for those that know how to use LLMs to learn on their own instead of just as a shortcut. And teachers who can deliver real value beyond what an LLM can provide will (or should) be highly valued.

[ericmcer]

It is probably a good time to view the root goals of education instead of the markers of success that we have been shooting at for a long time now (worksheets, standardized tests, etc.).

A one hour lecture where students (especially <20 year old kids) need to proactively interject if they don't understand something is a pretty terrible format.

> "Education seemed slow to adapt to the internet and mobile phones, usually treating them as threats rather than tools. Given the current incentive structure and the lack of understanding of how LLMs work"

Good point, it is less like a threat and more like... "how do we shoehorn this into our current processes without adapting them at all? Oh cool now the LLM generates and grades the worksheets for me!".

We might need to adjust to more long term projects, group projects, and move away from lectures. A teacher has 5*60=300 minutes a week with a class of ~26. If you broke the class into groups of 4 - 5 you could spend a significant amount of time with each group and really get a feel for the students beyond what grade the computer gives to their worksheet.

[gchallen]

As a teacher, I agree. There's a ton of covert AI grading taking place on college campuses. Some of it by actual permanent faculty, but I suspect most of it by overworked adjuncts and graduate student teaching assistants. I've seen little reporting on this, so it seems to be largely flying under the radar. For now. But it's definitely happening.

Is using AI to support grading such a bad idea? I think that there are probably ways to use it effectively to make grading more efficient and more fair. I'm sure some people are using good AI-supported grading workflows today, and their students are benefiting. But of course there are plenty of ways to get it wrong, and the fact that we're all pretending that it isn't happening is not facilitating the sharing of best practices.

Of course, contemplating the role of AI grading also requires facing the reality of human grading, which is often not pretty. Particularly the relationship between delay and utility in providing students with grading feedback. Rapid feedback enables learning and change, while once feedback is delayed too long, its utility falls to near zero. I suspect this curve actually goes to zero much more quickly than most people think. If AI can help educators get feedback returned to students more quickly, that may be a significant win, even if the feedback isn't quite as good. And reducing grading burden also opens up opportunities for students to directly respond to the critical feedback through resubmission, which is rare today on anything that is human-graded.

And of course, a lot of times university students get the worst of both worlds: feedback that is both unhelpful and delayed. I've been enrolling in English courses at my institution—which are free to me as a faculty member. I turned in a 4-page paper for the one I'm enrolled in now in mid-October. I received a few sentences of written feedback over a month later, and only two days before our next writing assignment was due. I feel lucky to have already learned how to write, somehow. And I hope that my fellow students in the course who are actual undergraduates are getting more useful feedback from the instructor. But in this case, AI would have provided better feedback, and much more quickly.

[whearyou]

“It's creating a strange feedback loop: students use an LLM to write the essay, and teachers use an LLM to grade it. It ends up being just one LLM talking to another, with no human intelligence in the middle.”

This was the plot to a recent South Park episode: https://m.imdb.com/title/tt27035146/

[HDThoreaun]

When I was in high school none of my teachers actually read any of the homework we turned in. They all skimmed it, maybe read the opening and closing paragraph if it was an essay. So I guess the question is if having an ai grade it is better than having a teacher look at it for 15 seconds, because that’s the real alternative.