[mnw21cam]

I teach at MSc level. My students are scattered around the country and world. This makes hand-written exams tricky. Luckily, the nature of the questions they are asked to solve in the essay I give them following their coursework are that chatbots produce appalling bad submissions.

[eks391]

This is great. Do you have advice on making questions that LLMs are bad at answering?

[mnw21cam]

In my case, I set some course-work, where they have to log in to a Linux server in the university and process a load of data, get the results, and then write the essay about the process. Because the LLM hasn't been able to log in and see the data or indeed the results, it doesn't have a clue what it's meant to talk about.

[red-iron-pine]

for most of the low hanging fruit it's as easy as copy-pasting the question into multiple LLMs and logging the output

do it again from a different IP or two.

there will be some pretty obvious patterns in responses. the smart kids will do minor prompt engineering "explain like you're Peter Griffin from Family Guy" or whatever, but even then there will be some core similarities.

or follow the example of someone here and post a question with hidden characters that will show up differently when copy-pasted.

[n4r9]

They don't work very well with large numbers. Try asking Claude to find the prime factors of 83521.