[chongli]

supposed to test for what? Your ability to synthesize knowledge out of thin air.

Yes, actually. Synthesis is the penultimate level of Bloom's taxonomy [1]. Perhaps not appropriate for a first quiz on new material, but absolutely in scope for a final exam.

I had many exams in university that challenged me to synthesize a proof to a hypothesis I'd never seen before. I had to bring together my knowledge of the material I'd learned in class with the bit of new material presented in the problem statement, and then devise a proof with one or more steps I'd never done before.

Many of my fellow students in first year struggled mightily with questions like this. There was much wailing and gnashing of teeth after the exams. In later years we'd all gotten used to it, and it was to be expected. That students in high school or younger years have been sheltered from synthesis-type problems on exams is a great disservice to them.

Edit: I should also note that outside of upper-year pure mathematics courses, synthesis-type problems rarely accounted for more than a third of the grade on an exam.

[1] https://en.wikipedia.org/wiki/Bloom's_taxonomy

[knollimar]

Synthesis problems and time pressure don't feel fair together.

I'd be furious to get some lofty theoretical synthesis problem on an engineering exam where the other problems are grindy analysis of "looks like concept you know but expand into a form you haven't seen and then will take 1/n time on your n question exam". I suppose these are already 1 extension in, so 1 more logical extension isn't that far off

[chongli]

They're absolutely fair if you're given plenty of practice at them. If the only synthesis problem you see all year is on the final exam then you have a point. However, if you're doing novel (to you) proofs on assignments every week throughout the course, you should be used to the novelty by the time you reach the exam.

Expecting a final exam consisting only of calculation and memorization type questions is wholly inappropriate in a mathematics course full of proofs.

[prerok]

I think you are both right and it really depends on the leap that has to be done during the time constraints of the exam and how much practise was included in the curriculum.

Off-topic, I know, but I always disliked the time pressure of exams. All the bad scores I got were down to not feeling quite right at the time of the exam: maybe I didn't sleep well, the food I ate did not quite sit with me, had a fever, etc. None of these things actually impair too badly but if the time is short, there is an impact.

I suppose it's similar to sports where you have to, at least somewhat, adapt on the go. Many teams botch a game here and there. But... your final exam? Botch it and you're screwed. It just does not feel right.

[hashmap]

it depends on what you mean by fair, i think. is the purpose of the test to gauge how well a person performs on tests? the difficult problems i encounter in the real world involve collecting a bunch of context on it, sitting with it for a bit, then putting it down and going for a walk. when i come up with something to try it's usually when i am outside and looking at trees.

maybe this is good for weeding out students who would not be a good fit for high stakes speed math.

[chongli]

It's really not high speed math. You'll have 2.5 hours to solve maybe 5 problems on the exam. A math grad student who TAs the course can typically complete the exam in 45 minutes or less, without any preparation whatsoever. An undergraduate student who can't solve the exam in 2.5 hours (without external aid) is not going to solve it in 10 or even 20 hours. They simply do not know what they're doing at that point.

It's really no different from a software dev interviewee who can't solve basic programming questions without being handheld and coached through the entire process.

[knollimar]

I think if TA no prep completion time to total time ratio is above 2.5 it's not the time pressure based exams I'm referring to.

I didn't get those exams in math courses. Engineering courses I did get those where some of the point of the test is the speed of your pattern matching ability (did you use design skills to reason about it or did you brute force a solution).

Math you're already extrapolating theory and you need ample time for that.

[menaerus]

I hope you don't interview people because what you laid out is full of false assumptions.

[hashmap]

Pointing to the interview process works against that argument rather than for it, and for the same reasons. Have you never been stuck on a problem at night and woken up with the answer? A good number of the really interesting problems I've solved got solved that way. Many of them felt like they either didn't have an answer or there's no way I was going to get it. Good thing there isn't a 2.5 hour timer following me around!

[chongli]

I have been stuck on problems that I figured out the next day. Many times. But that was when I was new to the topic, at the beginning of the course. By the time I reached the exam, I knew the tricks and how things fit together, and I could quickly solve lots of different proofs based on that same material.

That’s what the exam is testing for. If you need unlimited time to write the proof, then you haven’t studied the material yet, and should be expected to perform poorly on the exam.

It’s like the difference between someone who has never touched React before the interview, and someone who has been programming with React for several months. You expect the person with several months of experience to know how to solve problems with React several times faster than the person who doesn’t know it at all. If you’re hiring and looking for someone with a minimum of 1 year of React experience then you shouldn’t expect the person to take a full day to do basic tasks with React (especially if you the interviewer or anyone else on your team can do those tasks in just a few minutes).

[tekla]

If you can't do synthesis problems on a exam, you are not prepared for it.

I quite fondly remember a situation with a two question three hour open everything exam where I stared at the 1st problem and relevant equation for 20 minutes because I had no idea what I was looking at and was wondering if I should just leave and drop out.

Then I realized that a bunch of terms could cancel out and the problem became 3 lines trivial. The professor later admitted he wanted to give a "easy" question that you couldn't just copy out of your notes and had to think and to spend the rest of the exam on the actually difficult one.

[knollimar]

This seems like you proved my point by staring at it for 20 minutes before starting. I don't think it's fair to grade with extreme time pressure and also expect deep thought.

Being able to do deep thought quickly isn't a valuable skill, and you don't pick it up on the way ti mastery with lots of practice like you do the speed for rote problems

[tekla]

No the literal oppposite. The exam was "no deep thought required. Need to know the subject and its trivial"

[majormajor]

> Synthesis problems and time pressure don't feel fair together.

If you really want to grade the students its fair to put very hard problems on the exam to separate out mastery from merely very good understanding or recital.

[jltsiren]

Hard problems introduce noise to the grades. A student who gets a hard problem right could be good or lucky. In order to grade the students properly, you need multiple problems of similar difficulty. If you have both ordinary and difficult problems in the exam, it's probably long enough that it should take the entire day.

Undergraduate exams tend to be short. Which means that a perfect grade should be interpreted as "meets the expectations".

[chongli]

Which means that a perfect grade should be interpreted as "meets the expectations".

None of the mathematics exams I wrote during undergrad at Waterloo (as a math major) would fit that description. Nearly every single one of them had midterm grades with unimodal distributions centred below 70%, tending toward 60%. Typically, only 1-5 people in the class (of 100-200) would score a perfect grade. In upper year pure mathematics courses it was common to not have any perfect grades (in a class of about 20-30).

[jltsiren]

Mathematics is notorious for exams like that. But if you look at the reasons why people fail to get a perfect grade at undergraduate level, it's almost always due to honest mistakes or because they didn't learn what they were supposed to learn.

In my experience, studying mathematics is a bit weird. If you are ready to learn a topic, it's probably the field where you can get top grades with the least effort. But if you can't learn something with reasonable effort, hard work is unlikely to help. Doing something else and trying again after a few months might help.

[chongli]

If you are ready to learn a topic, it's probably the field where you can get top grades with the least effort.

Depends on your school. At my school, someone who was not super-talented in math but who works hard and is actually smart about studying is a 70s-80s student. The students who got 100s were basically IMO-level elite mathematics kids who were heavily recruited by the school and given full ride scholarships.

The course exams were essentially designed so that a "meets expectations" was a final grade of 65%. A grade of 100% is someone they were looking to recruit into research internships, grad school, and potentially a tenure-track position.

[knollimar]

Theres is more than ine dimension to mastery.

What skill are you grading? Can you expect some students to be able to churn out the problems? Are they doing design work? Is it more inventive or more creative? Is there more than one solution? How rigorous is the solution?

My point is that you can design an exam to test for speed and ability to do the day to day well. You prepare for this exam with lots of practice. There are a few problem types but you might not have seen every one. There's a basic level of extrapolation you're expecting.

You can design an exam for abstracting theory, and you prep for this exam by doing the hardest problems you can find, reading more theory, etc. You might not have seen anything like this but the test is to use the tools you're given in a class well. The test is the extrapolation.

I had theory courses where I expected the latter and practical courses where I expected the former.

There's more than one type of mastery; I bet your math professors would get destroyed by gifted 14 year olds in algebra competitions that involved arithmetic and timing. Do those 14 year olds not have more mastery?

[chermi]

Am I the only one that thinks a test in a technical discipline is poorly designed if more than one person gets a 100%? Exams are supposed to be hard. School is supposed to push you to learn the subject as best as possible.

[chermi]

I dread the day this collapse of thinking ability hits physics education. Maybe it already has.