[2ap]

I am an educator at a UK university. The essay is rapidly ceasing to be an appropriate way to assess students knowledge and critical thinking.

We regularly organise in person face to face practical exams for our entire several hundred strong year group of undergraduates. It is possible to do assessment properly if the will is there.

[dan-robertson]

When I was a student at a uk university, essays were, I think, written in examinations where one can’t easily use AI, as were most of the other things one would need for a degree. There was a small coursework component of my course and I think others. But I’m still a bit surprised — surely if you are getting a coursework essay from a history or English student or similar, the AI will be a worse writer and won’t have sufficiently detailed knowledge to write about the topic, so either the essay won’t really make any sense if you think about it (which you’d hope examiners would be able to tell) or the student will have done their own research and gotten the AI to form the students opinions into the essay, but they will then likely synthesise a much worse essay than the one the student could have written.

Doing practicals seems good, and I think proctored examinations can add objectivity too.

[2ap]

I agree - out of the box, a LLM will write an extremely poor essay. But that is not how to use the LLM. WHat you do, is give the topic to the LLM, and say, "with this topic, what are 20 titles that I could have for essays." Then, you pick the best title, and you then say, "I am writing an essay with the title <insert title>, what should the central argument for the essay be?" You then ask for a list of bullet points for the central theme of each paragraph to elaborate on that argument, then you ask it to write each paragraph individually, then you put the pieces together, and ask it to proof read it's own work and finally you add in some references of your own. Within 1/2 an hour, you have a passable essay. And if you then iterate on it with the LLM you can fairly soon have a half decent one, especially if it is a well trodden area (presumably with lots of training data that the LLM draws on).

I tried to get students to critique things, but even then you can put in the text to critique to a LLM with a long enough context, and the LLM will kick off with a passable critique, if you iterate with it enough.

So, even though they'll never get a top mark, they will still be able to get through the assessment. So I don't set essays any more.

[dan-robertson]

I guess I’m just surprised that the result is passable.