[jerf]

When I took my AI class, my professor would assign one paper a week and we had to turn in a "critique" of that paper. These papers were, of course, the seminal classics in the field, because what else would you want your intro class to be reading? I complained a bit to the professor that it was a bit silly to critique the seminal papers in the field and if you want to be sure we read it, we could just turn in a summary (which our critiques typically started with anyhow), but we were supposed to learn how to critique papers.

In hindsight, I suppose I should have appealed to the fact that critiquing the seminal papers in the field is a serious data set bias and that trying to "learn to critique" on the best papers ever written in a field was less likely to produce a useful "critiquing" skill and more likely to produce some overfitted garbage skill, but, hey, I hadn't taken AI yet! I didn't know how to express that.

(It did produce a garbage skill, too. I tried writing "real" critiques using my brain, but after getting Cs and Ds for the first couple, I learned my lesson, and mechanically spit out "Needs more data", "should have studied more", and as appropriate, "sample sizes were too small". Except for that last one, regardless of the study. Bam. A series of easy As. Sigh. I liked college over all, but there were some places I could certainly quibble.)

Anyhow, this is the paper that needs to be assigned towards the end of the semester, and students asked to "critique" it. It's a much better member of the training data set for this sort of skill.

[aidenn0]

Yeah, any "critique" or "argument" assignment seems to have weird grading biases.

In my freshman Composition class we needed to pick a controversial topic and argue one side. Highest grade on that assignment went to someone who argued that smoking causes lung cancer (this was 2003). The instructor explained that it was the most convincing paper. Those who picked an actually controversial topics got the lowest grades because their arguments were less of a slam-dunk.

[ViViDboarder]

I had to do something similar in a Physics class of mine. We had to write a paper critiquing and arguing against a well known physics theory. I was really into Special Relativity at the time, so picked that one.

My professor was a Nobel laureate, so already pretty intimidating. After we each picked a theory and wrote an outline he asked that we review with him. Well, he said, “I don’t know why you’d pick that. That’s going to be too hard.” So I asked if I could change, but he said that it’s too late for that and wished me luck… I was more than happy with the C+ I got.

[turdnagel]

Had a very similar experience in philosophy classes. The good thing about critiquing seminal works is that there's often a rich body of criticism already available. I found the only way to do well in those classes, since I lacked the mental horsepower to produce my own seminal critiques, was to parrot the literature that was out there already.

[jerf]

I think in philosophy it's a bit more valid. Still not maybe the best case, since these are still the papers that stood the test of time, critiquing some marginal paper would probably be a better practice of "critiquing", but philosophy always has a critique. The form may be impeccable, the writing may be spectacular, etc., but the philosophy itself always has room for rumination, discussion, etc.

For a science paper, if it's something people are still reading 30-50 years later, it was apparently good enough. I can always critique the paper for failing to solve String Theory and then draw out from String Theory a mathematical demonstration of how their solution for getting robots to navigate around boxes is very good, but that's more a reflection of me than the paper.