[avani]

Student reviews are also ridiculously sexist in the average case. In a recent study, the students rated identical teachers differently enough by gender that actual teaching performance was lost in the noise: https://academic.oup.com/jeea/advance-article-abstract/doi/1...

Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female professor, and their qualitative answers were also very different: https://www.insidehighered.com/news/2018/03/14/study-says-st...

I'm grateful to be a computer scientist, where the consolation prize for not getting tenure is a cushy industry job, but I feel deeply for my pre-tenure colleagues in the life sciences who have to work twice as hard for their evaluations while still maintaining their research and service requirements.

[YorkshireSeason]

   students rated identical teachers 

Did you actually read critically the papers you cite?

[1] claims to be comparing "identical teachers" but [1]'s sole claim to courses taught by male and female instructors is "neither students’ grades nor self-study hours are affected by the instructor’s gender". Clearly those are hardly the only factors relevant to teacher quality. Moreover [1] claims to use "objective measure of the instructors’ performance", and which includes -- I kid you not -- "self-reported number of hours students spent studying for the course".

[2] is similarly vague, and claims that "the courses were identical: all lectures, assignments, and content were exactly the same in all sections" only to to state in the next sentence that the "only aspects of the course that varied between Dr. Mitchell’s and Dr. Martin’s sections were the course grader and contact with the instructor". Well, isn't "contact with the instructor" significant?

Both [1, 2] use p-values [3], which doesn't increase confidence in the results.

As an aside, neither paper discusses potential bias the authors might have, in particular their own social desirability bias [4].

[1] https://academic.oup.com/jeea/advance-article-abstract/doi/1...

[2] https://www.insidehighered.com/news/2018/03/14/study-says-st...

[3] https://en.wikipedia.org/wiki/P-value

[4] https://en.wikipedia.org/wiki/Social_desirability_bias

[dalbasal]

I haven't, but.. commenting on the comments only

If it was possible to objectively determine teacher quality, student evaluation probably would not exist in the first place. The reason we ask people's opinion is in order to quantify the subjective. Double edged sword, because we tend to conflate quantified & objective.

That's half the theme here. erm.. people's opinions are subjective..

Part of the problem with "student-as-consumer" is that students aren't always the real consumer.

[snailletters]

> Please don't insinuate that someone hasn't read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."

[vonmoltke]

This has nothing to do with the article. This has to do with sources the poster chose as evidence to support a position. It is definitely relevant to question whether a poster has read and understood their own citations, particularly when those citations are not very good or contradict the poster.

[fogetti]

It's very relevant if the OP read the article or not. Makes his/her argument stronger/weaker because the articles are used as basis for the argument. So 'Did you even read the article?' is a valid question.

[verroq]

Surely it is worth the distinction when the contents of the article is being disputed.

[michaelt]

  [2] is similarly vague, [...] isn't "contact
  with the instructor" significant?

Well, they did compare online courses.

The article doesn't detail how their online courses are structured, but when I've done free udacity/coursera/edx courses, contact with the instructor has been nonexistent.

Seems to me it'd be very difficult to design a test to investigate this that wouldn't have some valid methodological criticisms. Even if you had lecturers, delivering online lectures with fake names, using voice-changing software and online-only office hours, you could still criticise that as being unrepresentative of real college lecturing, and having confounding factors if the courses ran at different schools, years, or times.

[craigzucchini]

From what I gathered from the mobile version (which may be paired down), the difference was not that significant, particularly in the analysis of evaluations and RMP submissions. Would have liked to find more on class composition, and would be curious how reproducible the relatively minor absolute differences would be, but "ridiculously sexist" seems a bit hyperbolic. Do you have a specific take on those studies that inform your opinion? I could be very well working with not enough information here.

[bufferoverflow]

What the hell are "identical teachers"? The premise alone doesn't make sense.