[kayo_20211030]

Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester.

How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term?

All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They can prepare as they wish, use AI, and even cheat on the homework, but there will be a revelation at the end of the term.

That final test can be proctored, monitored, audited to ensure that whatever words are used are indeed the student's own words. The resulting grade depends on that, and that alone.

The approach of continuous assessment, which to me always seemed suspect and ripe for abuse, was completely broken by the AI tools that are now available.

[snyp]

This approach does not really solve the core issue. In practice, students often do poorly when evaluation is concentrated in one end of term exam. It also pushes many students to cram at the end of the term instead of learning steadily.

A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that require original thinking and regular engagement can reduce incentives to cheat and improve learning outcomes.

https://www.sciencedirect.com/science/article/abs/pii/S22119...

[snapetom]

> Assignments that require original thinking and regular engagement can reduce incentives to cheat and improve learning outcomes.

At some point in college when I was thinking about law school, I learned about the Socratic Method. It was weird because up to that point in college, I just pretty much flew under the radar and took exams. It was far different than high school, and I realized my high school did pretty much use the Socratic Method. It wasn't as intense as law school, but every class, maybe 4-5 people would we grilled by teachers. This was called "participation."

Shy? Anxiety? Yeah, that didn't matter. Your number would eventually be up a few times a month. You had to prepare and know the assignments, otherwise your grade would suffer and public humiliation was a real thing.

[varjag]

It's a noble motivation (and not even unrealistic in 2015) but what you'll get would still be generative output.

If the only remedy is monitored end of term exams, so be it.

[snyp]

Err why can’t there be weekly monitored exams in class?

[catlifeonmars]

What exactly is that remediating? I don’t think that approach solves the problem of helping kids learn better.

[user205738]

Perhaps students should learn this information throughout the semester instead of on the last night or morning before exams?

If your goal is for them to know the entire material, then it makes sense to test their knowledge of the entire course in one exam, which also allows them to study at their own pace and order. If someone is unable to pass the exam or retain all the information, then consider whether you need such professionals.

[gmadsen]

then they fail. We are talking about adults.

[ufo]

Students are also people. If we're managing a software project, a single deadline at the end is sure to suffer from delays. It's better to split things into shorter deliverables with more frequent feedback.

[dghlsakjg]

You can have optional assignments and quizzes that serve that purpose.

If you take away the credit given for homework, you still can give feedback, while removing any incentive to cheat.

[ufo]

Few students do optional assignments unfortunately. Other tasks that are directly worth a gradetend to take priority (e.g. studying for another class that has an exam this week).

[dghlsakjg]

1. Class attendance is frequently optional, but students still attend.

2. I had a prof. that didn't require homework be done. He would give out "practice fun" and would gladly sit down, give feedback and 1:1 time to those who completed it, or tried. He also pointed out that it was rare to pass the exams for students who didn't do "practice fun". Most people did the work.

It leads me to believe - from my own experience too - that students generally aren't stupid, and will gladly do the work if there is a point. Plenty of homework is pure busywork though, even at the college level.

[DrewADesign]

I got diagnosed with cancer just before finals in the first semester of my senior year. Sure, it kill my chances at graduating Summa Cum Laude, and I didn’t make the Dean’s list that semester even though I worked my ass off, as usual. Frustrating, but that’s life. I should not, however, have failed that semester, which I would have if only the final week’s assignments were counted. People have bad weeks. In most white collar jobs I’d have probably been able to take some time for myself, maybe given someone else my most urgent tasks, and likely been given plenty of leeway. Even doctors, lawyers, etc. People deserve to have bad weeks without losing months of work.

[pibaker]

Let's just focus on CS for a moment because this is HN and I have a CS degree. How do you evaluate a student's ability to implement a piece of non trivial software following a specification that would take, say, over five thousand lines? This is a fairly common type of coursework for CS students. It's not possible to do it in one exam setting. Well, unless you can type continuously without having to stop to think or check your work.

What will likely happen if you have to force all evaluation into one exam is you will get a Leetcode style programming quiz, that tests something, except that something is only barely connected to your daily job after college. Or you end up with something like a multiple choice exam that is again disconnected from your work and can be gamed by studying just for the exam.

This exact line of argument that we should just abolish coursework keeps coming up on HN. Courseworks exist for a purpose. You can't just throw it away and pretend it never did anything useful.

[foxglacier]

A clever solution my professor did was we write the program in our own time, then take the code into an exam where we have to modify it to answer some novel question. It doesn't matter if we "cheated" writing the code as long as we understand it enough to change its behavior.

[armchairhacker]

Why not multiple exams? In fact, why not many exams?

Sure, it requires more resources, but it shouldn't require much more:

- We've had multiple exams before AI, and I don't see how AI makes it any harder. Obviously these are closed-book

- Schools should already be banning phones in class (and colleges have insane tuitions, they can afford more exams)

- The students who go out of their way to cheat - as long as they're a minority, let them. Why not? Either they'll fail later in life, or they didn't need to learn the material because they're pathological fakers (even if you won and forced them to learn the material, they'd probably still fake their way out of using it). Then, I doubt you need much proctoring to ensure that most students don't cheat, because most of the smart students are generally smart enough to know that actually learning the material is probably important (or if the material is probably not important, it doesn't matter if the students all cheat...)

Meanwhile, downsides of one exam:

- Disadvantages students who get overly stressed about unrecoverable exams, or have a particularly bad day on the exam

- Many students will blow off the (ungraded) assignments and put off actually learning until the end

- Less graded content (especially if the exam isn't overly long, which would disadvantage some students)

[BeetleB]

Indeed. Many of my technical undergrad courses were very exam heavy. Typically 3-5 midterms and one final. Sometimes the final was as little as 10% of the grade. The idea was that if you'd done well throughout the semester you can relax during finals weeks.

Homework was assigned but not graded.

Periodic tests is the way to go.

I hated courses where the final was more than 30%. Forget 100%.

[abathur]

I don't disagree with you that a reasonable way to cope with the current problems is to ensure everything that "counts" is done in a controlled environment, but pedagogy and its goals are vast.

There are things you learn from spending several days structuring a 20-page argument that you will not learn (and cannot assess) from oral examination or a 5-paragraph essay written in a blue book.

[kayo_20211030]

If you have spent several days structuring a 20 page argument in October on any topic you'll have learnt a great deal about the subject matter. When you get to the exam hall in, say, May it will stand to you.

That knowledge will show up in the blue book vis-a-vis the other exam candidates.

[abathur]

Sure--yes--the student will learn something if they actually wrote a 20-page paper on some given topic. But how are you going to evaluate their ability to compose the 20-page argument?

I would prefer not to be confrontational here, but I am having a hard time imagining that you've deeply considered the pedagogy of how to teach and evaluate students on squishy skills like this.

Knowing a bunch of facts about something is a world apart from structuring a compelling in-depth argument about it.

[kayo_20211030]

In the simplest case, where we'll say the exam question was precisely the topic of the 20 page paper, the candidate would be golden. Of course, it's unlikely in a 3 hr. exam that you'll be asked to write a 20 page response; but in edited form, you could definitely produce three cogent pages about some particular aspect of the original paper - if you've done the work. If you truly wrote the 20 page paper, you can surely produce three literate, cogent, responsive and topical pages.

[Wowfunhappy]

I think schools need to set up additional, new proctoring sessions for this type of work. This will likely be something they have to hire for. A student can come and work for four hours, then hand in their in-progress draft and leave, then return later to finish it. (And please for the love of god, let students do this on offline computers, don't make them handwrite everything!)

[pibaker]

What stops a student from going home and asking chatGPT to write about few bullet points about how to answer and coming back the next day doing what chatGPT told him to do?

[Wowfunhappy]

They'd need a very good memory. You can't make cheating impossible, before ChatGPT they could have asked a friend for help.

[ipcress_file]

This assumes that the assignments and the exam cover the same material. That's not always the case.

[kayo_20211030]

That would be really poor course design :-)

[abathur]

There are many disciplines in which students work on effectively distinct projects.

For example, the life-changingly-well-designed newswriting course I took in college assigned every single student a different story to spend several weeks reporting out so that we wouldn't all be out harassing the same poor people for interviews.

[kayo_20211030]

Genuinely interested. What was the final like? This seems more in the experimental science (ok, journalism) category. I may have to adjust my thinking to be more expansive and also include things like "vocational".

[b800h]

But there's no reason to expect that work to be graded. It should be a learning exercise which trains skills later tested under exam conditions.

[robotresearcher]

Many students will simply not do these assignments. They should but they won’t. Continuous assessment partly solves this.

[eudamoniac]

Schools stopped doing that because students largely refuse to prepare. Testing throughout the year is like a CI pipeline and is shown to work better for the median student.

[kayo_20211030]

Students are neither generally stupid nor constitutionally lazy. I sense that when expectations are clear they'll often surprise you with diligence. We should trust them to do the right thing. If they do, it's an A; and if not, it's less than that.

[csa]

> I sense that when expectations are clear they'll often surprise you with diligence.

Data does not support your sense.

Most students do not have good time management skills, usually because they have no models and/or have not been taught these skills.

Furthermore, continuous feedback, whether graded or not, has been found to be more effective than one-shot feedback.

Evaluation and assessment is a complex topic towards which many people (not necessarily you) want to take an overly simplified approach.

There are trade offs for any system that is chosen. The organizations providing the grades have to decide what their priorities are (e.g., time, accuracy, etc.).

[eudamoniac]

I'm not sure what public school system has instilled that confidence in you, but it musn't have been mine. I'm also not sure why you think clear expectations about an end of year test will lead to better results than clear expectations about multiple spaced out tests. The data shows that it doesn't.

[mikkupikku]

I think if they offered a proctored do-over a week later, the bad results on the first test might prompt them to make an attempt at studying for the next week, and the prospect of having to sit through two tests and getting shamed for having to do-over might prompt people to actually study for the first test.

[j-bos]

Throw in some oratory presentations as well and that sounds like a curriculum.

[kayo_20211030]

Yes. That's fair. Students should know, up front, what they're supposed to learn.

[ufo]

Students are very grade-motivated and unfortunately they rarely do the homework assignments if they are not worth points.

At-home coding projects, writing essays, etc also exercise different skiils than you can test for in a 2 hour written exam. It's unfortunate that due to rampant AI cheating, we can no longer reward the students who put in the work and develop these skills.

[limagnolia]

Why a single test at the end of the semester? Why not allow the student to demonstrate mastery at anytime during the semester when they are ready? Then they can move on to the next objective, or, if they fall short, continue to study until they meet the objectives.

Of course, creating good exams is difficult, but you have to do that either way.

[robotresearcher]

Unfortunately this approach is known to be really bad for some students. Continuous assessment is bad for other students.

I was lucky in that my education last millennium was almost entirely make-or-break final exams, which suit me well. I’m bad at routinely completing little assignments, but shine under crazy pressure. Other students are the opposite.

We’ve tried both kinds over time and trend. Neither is perfect.

[khazhoux]

> one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. The resulting grade depends on that, and that alone.

I love this idea. And if a student is having a really bad day, or their dog just died, or they have bad cramps, or they have a hard time dealing with the intense stress of your entire grade being decided in one exam... well, those loser students can just fuck right off.

[kayo_20211030]

Would you design a system to assess knowledge, avoiding the distortions of AI on weekly submissions, according to the general case or the exceptional case?

Accommodations are part of the fabric already. It doesn't seem inconceivable that we couldn't deal with them in exceptional circumstances in a similar way to how it's done today.

[reshlo]

How it’s done today is that they rely on your other marks from earlier in the semester to inform how your exam grade should be adjusted. That doesn’t work if there are no other marks to use.

[kayo_20211030]

Yes. There will be no other marks available for adjusting a final grade. One test, one mark. One knows the stuff, or one doesn't.

Accommodations are real and necessary, but applied at the end.

(Experimental sciences are an exception)

[khazhoux]

How do you propose to avoid "distortions of AI" in your mega-examination?

... well then, why not use those same protections (proctoring, monitoring, auditing) in continuous examination?

[user205738]

If a student knows how to communicate, they can solve this problem: warn the teacher, take a sick leave if they feel they are not ready, and other options.

If they did not do this, they failed the exam on communication with other people.

In addition, we are not always able to make decisions in ideal conditions. We need to learn how to solve problems under pressure, in emotional turmoil, and when we are not feeling well.

[MaKey]

That's how it was for me - one exam per course at the end of each semester. To qualify for the exam you had to do take-home assignments. Didn't pass? Try again next semester. Was it easy? Hell no, but I learned a lot.

[upboundspiral]

The purpose of grades is to punish students, something which they are keenly aware of. Remove grades from the equation and hold students back until they have mastered the material and they will cease cheating.

If someone knows 80% of the topics on an exam like the back of their hand and doesn't know the other 20% they shouldn't get a B, they should pass the subjects they know and be asked to retake and relearn the subjects they don't know.

When people know they can make mistakes and the result is not a perpetual black mark on their record (any grade not an A) but they are given the chance to improve and demonstrate this improvement then perhaps they might be more willing to admit and understand mistakes instead of cheating.

[thfuran]

What exactly would the goal of this change be?

[kayo_20211030]

Ultimately, you ask the student, in one audited test, to demonstrate that they've absorbed the essence of the course material and have developed some level of mastery.

[thfuran]

Okay, so the system is designed not to educate but to minimize the time required to determine whether students somehow stumbled into an education?

[jazzyjackson]

???

Do you only learn when you’re being graded?

[Retric]

If the change is not designed to educate the student, then the point isn’t education.

As a general rule when changing complex systems, you sacrifice what you aren’t trying to optimize. If you make a random change to a car without consideration for gas mileage it’s very likely to reduce gas mileage.

[jazzyjackson]

Schools are not merely in the business of maximizing education, they have their own prestige to uphold, and they would like to give degrees with their name on it to students who have actually upheld their end of the contract.

(The other side of that contract is, kids are not merely attending schools to learn, but to earn a degree that carries some degree of prestige)

[jazzyjackson]

To have end of semester grades be determined by work that is done by the student, not through weekly assignments where it’s trivial to cheat

[thfuran]

To what end? Not cheating on the weekly assignments is surely more beneficial to learning than cheating on them is, but I don’t see how removing the assignments altogether would help students learn.

[kayo_20211030]

If you nail the one exam, you get an A+. If you fail it, you get an F. In between, you get what your score says you get.

[thfuran]

I understand the proposed grading system but not the reason for selecting that particular system.

[kayo_20211030]

It's a crude blade to avoid the issues of AI pollution of weekly submissions, of which few teachers have much confidence that the submission itself was actually written by the student - who's assumed to be learning something.

The OP was about students dumbing down their own work to avoid AI detectors ratting them out. That seems like a big loss.

[jupp0r]

And what would the goal of that be? I thought the goal of education was... education. The grading is not goal in itself. Will this really motivate kids to do better?

[kayo_20211030]

It's to prove that a student is actually educated and has a firm grasp of the course material. If one gets an A every week on AI-assisted submissions, can one make such a claim? And can a teacher make the claim that they've achieved any actual education of the student?

A grade, on a single proctored test, is a crude metric, but at least it would be a brutally fair one.

[naveen99]

Maybe you can call it an interview…

[b800h]

Came here to say the same thing. The AI problem is functionally no different to the paid essay writers. Grade everything at face value, and then have people write essays under exam conditions for grading.

[nz]

Even before LLMs, there was a _lot_ of deception and cheating in university. I -- and I do not say this with pride -- used to write essays for my classmates for money. In my own defense, I needed the money. I also know that in addition to homework for money, many fraternities and sororities kept copies of prior exams and assignments, and getting access to these was one of the perks of membership. Knowing what kind of questions to expect (let alone the exact questions) can easily give someone a few extra IQ points for free.

Personally, I felt that the drive to automate the parts of the professors' workloads that mattered (i.e. teaching and grading and evaluation and research), only so that they can be given work that matters less the more they do it (i.e. publishing slightly different flavors of the same paper, to meet KPIs), was oddly perverse.

The multiple-choice test and the puzzle-solving test and really any standardized test can be exploited by any group that is sufficiently organized. This is also true in corporate interviewing where corporations think (or pretend) that they are interviewing an individual, whereas they are actually interviewing a _network_ of candidates who share details about the interviewers and the questions. I know people who got rejected in spite of getting all the interview questions correct (the theory is that nobody can do that well, so they must have had help from previously rejected/accepted candidates).

The word "trust" shares a root with the word "tree" and "truth" and "druid". Most exams and interviews are trying to speed-run trust-building (note that "verification" is from the latin word that means "true"). If trust and truth are analogous to "tree", then we are trying to speed-run the growth of a tree -- much like the orange tree, in the film, _The Illusionist_. And like the orange tree, it is a near-complete illusion, a ritual meant to keep the legal department and HR department happy.

The LLMs have simply made the corruption of academia accessible to _all_ students with an internet connection (EDIT: and instantaneous and cheap, unlike a human writer).

There has never been a shortcut to building trust. One cannot LLM their way into being a (metaphorical) druid.

I do not look forward to the Voight-Kampff tests that will come to dominate all aspects of online and asynchronous human interaction.

Note that, short of homework/classwork that _can't_ be gamed by an LLM (for some fundamental reason), even the high-quality honest students will be forced to cheat, so as to not be eclipsed by the actual low-quality cheating students[0].

I imagine that we may end up wrapping around to live in-person dialectics, as were standard in the time of Socrates and Parmenides[1]. If so, this should be fun.

[0]: If left unaddressed, we may see a bimodal distribution of great and terrible students graduating college, with those in between dropping out. If college is an attempt to categorize and rank a population, this would be a major fault in that mechanism.

[1]: Not to the exclusion of the other kinds of tests, writing is still important, critical even. But as a kind of verification-step, that should inform how much the academic community should trust the writing (I can imagine that all the writers here are experiencing stage-fright as they are reading these words).