[dual_basis]

When I was a college instructor I found that when I tried hard - putting a lot of thought into pedagogy, having weekly (open notes) quizzes, assigning challenging but fair homework - I got terrible evaluations, even though the average grade in my class was higher than other teachers who were teaching the same course. When I phoned it in, didn't really try, was very lax with the homework and often made the quizzes take-home, I got excellent reviews but the average grade in the class was worse.

[xtrapolate]

> "When I was a college instructor I found that when I tried hard - putting a lot of thought into pedagogy, having weekly (open notes) quizzes, assigning challenging but fair homework - I got terrible evaluations, even though the average grade in my class was higher than other teachers who were teaching the same course. When I phoned it in, didn't really try, was very lax with the homework and often made the quizzes take-home, I got excellent reviews but the average grade in the class was worse."

It must've been a frustrating experience.

At the same time - you're making a very broad statement here based on a rather personal experience. You went from a certain regimen yielding certain results, to a different regimen yielding different results. There are way too many parameters here to draw conclusions.

[dual_basis]

Sure, but that's the sort of data we have to deal with in this arena. Who's going to run a large randomized trial where students are purposely assigned to different classes (keeping in mind that schedule conflicts already add additional constraints to this which may bias these assignments) and then, furthermore, have the instruction fixed apart from how easy the assignments are? Is it even fair to the students to knowingly assign students to relatively poorer teaching? Clearly it happens all the time, every department has that professor who is known for being a bad teacher yet they still have to assign classes.

In speaking with other grad students this seemed to be a well-known phenomenon, to the point that most other grad students intentionally didn't put much time into their teaching and basked in the positive reviews as a result. It was suggested many times to me that I was spending too much time thinking about my teaching. In my case, the lax teaching was not intentional, I simply was overcommitted that semester and had less time to prepare.

[thatcat]

Did your reviews not ask for justification of the ratings?

[dual_basis]

There are a bunch of ratings 1-10, and then a place to optionally add comments. It's hard to consolidate information from 200+ comments, but, then again, most students do not fill out the comment section anyway.

(In a sense, I guess the whole thing is optional - after all, it is anonymous, and we don't check that every student has filled the whole thing out. It also switched from in-class to an online form during the time I was teaching, and the response rate went down quite a bit as a result, but this was after the situation I mentioned above.)