[beisner]

The idea is that you create questions that are ungoogleable. Most of my upper-level CS courses had take-homes that were open-book, open-internet, and the professor basically said “good luck finding the answer to this problem online; you will not”.

This is harder for professors to do, and maybe fails in intro courses, but generally is quite effective.

[BalinKing]

One of my Ph 1b (freshman special relativity and electrostatics) quizzes guided you through "discovering" a new method of solving that type of problem. I don't remember the specific details, but it's one of the rare tests I've taken that I actually enjoyed!

Although most homeworks and exams aren't nearly that cool IME, they're virtually all about synthesis and fundamental understanding rather than rote memorization. This "first principles" approach to most everything is one of my favorite things about Caltech.

[tzs]

When Carl Sagan was writing "Contact" he attended a conference that was also attended by his friend Kip Thorne. Sagan knew that one of Thorne's pet peeves was science fiction that just hand-waved away the physics of FTL travel.

Sagan told Thorne he was writing a science fiction novel, needed FTL travel, and asked Thorne if he could suggest something that would be reasonable. Thorne agreed to look into it.

After the conference, Thorne spent a while working on it and came up with a wormhole approach and worked out the physics of it.

In addition to giving it to Sagan, Thorne also put it on the Ph 236 (General Relativity) final exam. He didn't tell the students on the exam that it might imply FTL travel. He just set up the conditions and had them work out the physics. Most of his students succeeded in that, but he was a little disappointed that none of them happened to notice that it implied FTL travel.

(I got the above from an unpublished book Thorne was working on in the early '80s. It was a collection of biographies of and interviews with physicists, astronomers, cosmologists, etc., written and conducted by Thorne. He had the draft chapters and the raw interview transcripts in a world readable directory on the physic's departments VAX, where they were widely read by the rest of us with accounts on that machine).

[aschatten]

Do you know if it's possible to get ahold of a copy?

[tzs]

As far as I know, he never finished it, so unless he still has and would give out a copy it is probably unobtainable.

[908B64B197]

This.

Else you might as well call the degree "rote" because that's what you measure.

> maybe fails in intro courses

If someone cheats their way through intro courses they are up for a bad surprise since the upper level courses assumes a mastery of the intro classes!

[_8091149529]

Interesting, thanks for the replies.

I'd still point out that the system is not robust against: - Working in groups. (Bad if tests are supposed to assess individual performance.) - Asking outsiders for help.

Also, your downplaying of "rote" learning feels misguided, no matter how advanced/abstract/high-level the domain in question is. Cue the Bruce Lee quote about 10,000 kicks..

[908B64B197]

> Working in groups.

Students hate carrying dead weight so to speak. Sure they might be tempted to swap favors (A helps B for a certain subject and the reverse is true for an other one). If the exam is properly constructed you might be able to detect plagiarism however.

Rote is absolutely necessary for a well rounded education, but I really feel the need to overcorrect in the opposite direction because testing for rote is the lazy approach. Over time a lot of testing has the tendency to shift to simply measuring rote.

[turbinerneiter]

Just imagine a world were solving problems in groups and asking for outside help is bad.

We are so used to exams being about doing stuff that literally nobody in real life would ever even consider.

[_8091149529]

I was aware of the implications, hence the disclaimer "(Bad if ...)"

However, I'd assume that doing either during a test would be against the honor code.

[maerF0x0]

I had a professor that intentionally asked a question which wikipedia has the wrong answer... Called out and shamed everyone who used wikipedia as a source.

[dghughes]

That reminds me of when I took and distance learning electronics technician course. You could Google (search) all you wanted to but seeing answers didn't help in knowing how to solve the equations.

[jackson1442]

I think the most important part of "proctoring" exams should be that it is individual work. Let's face it- you will almost always have access to the internet. The only thing that should matter is that the assigned student can solve the problem _in some manner._ This may include the internet, notes, or really anything.

The real problem is people paying others to take exams, or simply copying answers from others. I work as an independent tutor, and I get contacted far too much by people looking to have me take their exams for them.

[asdff]

That was certainly a problem in person too. At my undergrad, for larger exams you would just pass your ID down the row to the person in the aisle and a TA would collect them all then check everyone in. Wouldn't be hard to just hand your ringer your ID to pass on up. Not unusual to look nothing like your freshman year ID photo in person.