[cloud5ir]

To me, this underscores notion that schools continue to test for the wrong thing. With the exception of professions/trades that require impulse application of knowledge, I don't know why I would want/need someone to memorize a concept in order to apply it. Productive members of our society/workforce think critically, they ask good questions, and they research and leverage the information at their disposal to make data-driven decisions. Let's test for that.

[sn41]

I teach programming. What prevents students from colluding over their tablets/mobiles/laptops while taking the exam from another device? Can't the student mirror the screen elsewhere, leading to interesting possibilities for cheating?

Proctoring is not a solution that I am comfortable with. I do not want to peer into the private lives of students and their home environments. Not every student is well-off, and has a private space all to their own for 3 hours.

I think taking tests from home does not really work with any of the models I have seen discussed this year. Cheating is real. It has nothing to do with rote learning. I am out of ideas which are foolproof.

[ouid]

The main problem, of course, is that receiving a credential and receiving an education are very different things that we do at the same time out of tradition and convenience. When testing was easy, it made sense to lump it in with teaching, but now that testing is impossible, I say we ditch it for the time being.

[gravypod]

Could you make your class project based and give everyone different, but equally difficult, projects of their choosing. Each check in into master would then be merged after a code review with you where you ask questions about why implementations are done specific ways to solve the problem/refactor. Code is only merged when it hits a quality marker you set. Students are graded by percentage of deliverable hit from original project specs.

Ex: student is to implement a K&R C compiler + code generator in 1 semester. Help them break up the project into manageable chunks (tokenize, AST, register allocation, etc). Each feature includes tests, comments, docs, and is presented to you by X date. Code review and cycle as many times as needed as long as it gets in by X date. By the end of the class the student hit 100% of their deadlines but only pass 70% of some test C files you create so they get a 70% (or higher if their code was consistently good and did not need cleanup) in the class or something.

It would never work because "you need to have a final exam" but it's the ideal I think we should strive for if you are "teaching programming" since this is how most work I've done in the real world has gone.

[acbart]

This will never work because it absolutely fails to scale, at a time when we have a complete lack of computing educators. Customized projects for each student? I have 159 students in my CS1 this semester, and I can barely keep track of them with the massive amounts of data I collect. The idea of making individualized assignments is beautiful but unrealistic.

[gravypod]

Most code reviews I've done at work take between 5 minutes to 10 minutes. To assume the worst possible case of 200 students, 20 deliverable per student, and a 10 minutes meeting per MR you'd be at ~28 days (~84 8hr-days). Get 1 TA and you now have 42 days of the semester dedicated to 1:1 time (35%) and the remainder of the semester dedicated to producing learning content. Learning content (lecture videos, etc) can be produced ahead of time and in this case you can replace this with more 1:1 tutoring time effectively allowing you to give each student ~10hr (10 min * 20 sessions * 1/.35) direct 1:1 time over the course.

From the students perspective all they need to do is:

1. Write a "project proposal" where they define a project, define goals, schedule a checkin with you to see if that project is large enough (1/20th the grade of the class)

2. Watch lecture videos and go through exercises on their own time.

3. Write code and bring it in to teacher. (18/20th of their grade)

4. Demo their finished project to professor (1/20th their grade)

This does work for later classes. I never had the ability to test this workflow out for earlier (100-level) classes as all of those classes I TA'd for followed the same model you're talking about.

The 200+ level classes that followed the model I'm talking about had ~10% to 30% pass rate which was in line (~2x) with the pass rate of similarly leveled courses from our college's Math and Physics department which was used as a sanity check.

edit: that's not to say you're incorrect that it would be difficult to do at scale, this is just the way I've seen it done at moderate scale (20 to 40 students).

[unishark]

It's easy (even a bit fun) to grade and give feedback for good students. Meanwhile it's a huge pain to do for the weak students. You need to first figure out what they heck they're trying to do before you can tell them how to fix it, which may take a lot. I suspect code reviews for professionals is much more like the good student category.

If you're asking the students to define unique projects themselves, you run into new problems as students dig up the most obscure blogs they can find on the internet to download and give you. It's a never-ending adversarial struggle.

[avh02]

You'll be surprised at what "weak" code you'll see in the real world and have to explain to someone it's not maintainable or clean or whatever.

This is especially true of junior data scientists (in my anecdotal experience so far)

[quiescant_dodo]

Code reviews I have done for unrelated bits of code often take me 90 minutes. And that's for code that fits into the idea of my product space, but not super familiar to my everyday life. I don't think 5-10 minutes is anywhere close to reasonable.

[a-nikolaev]

The hard part is giving the student meaningful feedback. Especially to those who don't understand something important. It takes time to understand what they do and don't understand, and explain it at their level.

[scscsc]

I teach classes of ~500 students and classes of ~100 students.

While having individual projects is very nice and it works for smaller classes, it does not scale to average students.

I have once used individual projects to grade the ~100-student class and the amount of effort it takes is unbelievable.

For example, just your number "1." point, write a project proposal, is not so easy. It takes hours to write a good proposal and several back-and-forths between instructor and student to agree on a reasonable (not too advanced/not too basic) project.

[dmurray]

10 to 30% pass rate? And that was very high compared to other departments?

Do you mind sharing roughly where this was (country and quality of university)? I thought we were in an era of grade inflation where everyone gets an A.

[gravypod]

This was course a 300-level test course compared to a 300-level test course from these other departments. The hope is that when they become "real" courses the pass rate is higher. I think this counts people who are re-taking the course or who dropped it as failing. We had ~50% of our class drop within the first month.

Part of it was related to their expectations of the course: "you mean I have to write code and the class is based off of that and not memorizing test answers? I'm out"

[unishark]

I think it can be automated in the more basic courses at least. But wouldn't solve the problem of students getting someone else to do the project for them anyway.

[FuckButtons]

So much this: cs61a at Berkeley for example has almost 2000 students in a single class, the idea of personalized anything with that many students is total unfeasible.

[Findeton]

I thought in the US people paid money for going to university, so that classes are well-funded enough... /sarcasm

[FuckButtons]

Berkeley is a public university, so many students qualify for federal aid and wind up paying very little or nothing at all,the full cost of in state tuition to those who can afford it is about $6000 per semester.

[LudwigNagasena]

This is ridiculous. Why do Americans pay thousands of dollars for a bit fancier version of MOOC? When I studied we had 200 people on a course sitting on lectures, 30 on practical sessions and everyone could have a personal session during the professor’s office hours.

There is some kind of a massive waste of money going on.

[poulsbohemian]

I'm genuinely curious to learn your option on cheating - does it actually matter?

What I mean is - certainly you don't want to harm the reputation of your school, and you want your students to have actually learned the material and be prepared for whatever they do in life that requires that knowledge... but what I'm wondering is... if there's presumably some tiny percentage of a class that is going to actually cheat or act in ethically questionable ways, are we in fact putting more effort into the prevention of that cheating than is warranted?

[unishark]

One third of the students on my last exam cheated (at least, these were the ones that gave an identical answer to one of the problems, including identical minor typos). I was using the tools described in the article, but apparently I was being too lax about enforcing the 360 degree views or something as they clearly colluded to find a way to share answers anyway.

[eatthoselemons]

I feel like this hits the real problem, figure out a better way to prevent cheating and a better cheat will be made.

It should instead be worked on making cheat resistant material

(unfortunately I don't know how to do it but I feel like the later is where the energy/brainstorming should be spent)

[nl]

> presumably some tiny percentage of a class that is going to actually cheat or act in ethically questionable ways

According to the International Centre for Academic Integrity, from 71,000+ US students surveyed between 2002 and 2015 at least 68% of students admitted cheating in some way.

https://www.academicintegrity.org/statistics/

[cute_boi]

Other 32% lied /s

[alach11]

Of course it matters. In a class where cheating is permitted, the students who choose to cheat will perform better than they otherwise would have. That will give them an advantage over the students who choose to not cheat.

I personally experienced this in undergraduate with a professor who chose to ignore obvious cheating and I think it negatively impacted the motivation of students to learn the material.

[raxxorrax]

Probably out of scope due to time constrains, but having students explain solutions and code should give a glimpse about how well they understood the principle. So if you have some time to kill...

[TheAceOfHearts]

How about actually treating them like respectable adults and using the honor system? If you start treating people like dogs, don't be surprised when they bite you. Woof.

Stop giving bullshit tests with bullshit restrictions, which is usually what drives people to cheat in the first place. Have you considered asking: what can I change so my students are able to successfully learn the material such that they won't feel compelled to cheat? Recognize that cheating students are a reflection of your apathetic educational strategy and ineffective testing model.

Here's an example of a bullshit test which exemplifies a few of the issues with apathetic educators: single-try tests. Students are only given a single attempt to demonstrate they've learned the material, with the result ultimately reflected in the final grade. The apathetic educator is unbothered by having failed to adequately prepare his students, despite his knowledge of their inability to accurately self-asses in an unknown topic. A caring educator would provide a way for students to diagnose and fix any gaps in their understanding in addition to as a path for them to raise their grade to an A if they demonstrate an understanding of all relevant course topics.

[smabie]

Memorization is absolutely crucial to getting good at math. In fact, I don't think there's any other way to get good at math besides memorization and doing problems, over and over and over again. It's rote, sure, but it has tremendous value.

If you think you understand something, but can't actually solve problems without referencing anything, you don't actually understand it.

[3PS]

I have a degree in math and I strongly disagree with this comment. I was always terrible at memorizing things, and I still regularly reach for Google to double-check formulas and theorems I should know well. In practice, I think a lot of math people end up memorizing things the same way programmers memorize the syntax and tools of their preferred languages; it's not necessarily something you explicitly set out to memorize, but you use them so often that you end up internalizing them anyway.

[eikenberry]

Not going to argue math, but that final statement is a poor way to frame understanding/learning. You can understand something well enough to solve it from memory a few times when in close proximity to the time you learnt it. On the other hand you can understand something well enough to know how to check the references and solve it for the rest of your life.

That is it is better to understand how to solve problems with references than without. You'll forget most you know but once recorded you can't forget. Then you just need to know it exists so you can find it.

[ylyn]

This is absolutely backwards.

No amount of reference sheets will help you with mathematics if you don't understand something.

[smabie]

I think you have backwards: you only understand something after grinding out problems and memorization.

[arpa]

Anecdote time. Never have I ever understood concepts through repetition alone. Grinding problems is for building patterns via using different input sets, not unlike ML. It is true that it "clicks" sometimes after a threshold and you actually understand it, but usually what grinding does is reinforce the algorithm without actually understanding it - e.g. surely you know that c squared equals a squared plus b squared, you've probably grinded that enough to memorize that equation. Can you prove it or at least explain why that equation is correct? Because i can't, not even after repeating it a million times. That's the difference between memorization and understanding.

[centimeter]

> Memorization is absolutely crucial to getting good at math.

I have a terrible memory and I excel at math (and majored in physics in undergrad) precisely because it doesn’t require memorization.

I don’t know anyone who is good at math who operates by memorization.

[Tainnor]

I think this is only half true. Of course, if you want to be good at maths, you will have to remember things because you can't always look up or re-derive every smallest detail (and because you have to become good at pattern-matching at some point). But the memory tends to work best if it's activated by deliberate practice. For example, instead of memorising a proof line-by-line it's much better to try to remember the key points and then practicing filling in the gaps. If necessary, repeat this process several times. When I was studying for ODEs, I tried to re-derive many of the standard techniques by trying to remember which trick to apply instead of trying to memorise the formula.

[KMag]

^ This. For my Mechanical Engineering undergraduate courses, almost all of the formulas (models, actually) could be re-derived from linear approximations on infinitesimal elements. Memorize and internalize how the model is derived, practice re-deriving the model, and you can re-derive the formula during the exam, and do a bit of hand-wavy intuitive double-check of your answers. If you've only memorized the formula, you're going to have a very tough time coming up with an estimate against which to check your formula's answer.

On a side note, I really wish we had more emphasis on the conditions under which the linear approximations broke down. I remember sitting at the front of the MIT 2.002 class, and there was a demonstration of metal fatigue using a hydraulic press at the back of the classroom. Professor Sanjay Sarma stepped up to the front row in order to better see the demonstration at the back, and so I asked him about my intuitions about which way the model diverged from reality under vibration frequencies high enough that the quasistatic assumptions built into the model broke down. He looked to both sides of us and told the students on either side of me not to listen because they might get confused, and then we had a little discussion about conditions beyond which the model applied and which way the model's error went under those conditions. It was simultaneously one of the best and most disappointing moments in my education. It was an exciting discussion, but I was sad that the world beyond the linear approximations was considered to be likely too deep a rabbit hole for most of the class. Sanjay (as he preferred to be addressed) was an excellent educator, and I'm sure his judgement was based on past experience... each semester has a given complexity budget, and the field of Mechanical Engineering is so broad (statics, dynamics, thermodynamics, fluid dynamics, mechanisms, control theory/sensors/OpAmps, manufacturing techniques, design for mass manufacturing, numerical process control, destructive/non-destructive testing, etc., etc.) that undergraduates need to spread a limited complexity budget across so many subjects that they can each only be covered relatively shallowly.

[Green_man]

I'm very curious about the type of math you have learned this way. Every math class was heavy on repeated exercises and memorization, but real understanding didn't come until outside of class (or after the semester) where reflecting on larger ideas lead to mental connections and analogies. I'll freely admit that I have to look up the integral/derivative of simple trigonometric functions every time, despite every attempt from the instructor to hammer those in during Calc II. Doing hundreds of integrals on homework and study guides kind of worked to learn those rules, but only imperfectly and temporarily. Doing hundreds of exercises over and over again has been even less useful for my later math classes. If some proof doesn't make sense, the next step is not to go through 100 examples, it is to break down the pieces into logical structures that are recognizable. All real understanding of math fundamentally works this way. (disclaimer, my math past calc/linear algebra has been in CS classes, so I'm definitely open to hearing how those from other backgrounds or those with more formal math education disagree with me)

If you think you understand some mathematical rule, but can only show 100 problems from memory, you don't actually understand it that well. If you can convincingly show why no counterexample exists, then you understand it in the strongest possible way.

[watwut]

Doing the same problem again and again does not do much to learn math. Doing various problems again and again is necessary, but that is not memorization. Because you are not actually remembering things.

Also, it was not that unusual for exercised based university math tests to allow references. Precisely because it does not matter all that much for difficult exercises, remembering everything is not the point.

[throwaway3699]

Is there not an in-between? I used to be very good at maths, but a year out of practice and I might struggle at some topics. I still retain the intuitive understanding and that grants me a rapid path back to being able to solve problems.

[whoisburbansky]

The intuitive understanding that you still retain was only built up by repetition and memorization in the first place, no?

[centimeter]

Absolutely not. I honestly can’t identify with your comments in the slightest. The only part of math that I ever learned by memorization was addition and multiplication tables in elementary school. Every subsequent revelation was based on pure understanding - zero repetition required. Up to what level of math have you studied? I honestly can’t imagine anyone thinking they’re good at undergrad-level math if they do it by memorization rather than understanding.

[weehoo]

The great grandparent to your post (the one who initially brought up the necessity of rote learning) is a quant at a market maker.

Anecdotally, my favorite professor from undergrad (born in China, PhD from the best department in his field [my personal opinion]) said he thought the reason for Russian/Chinese dominance in certain areas of math was due to how those areas benefitted very much from rote practice. He advised all of us (American undergrads) to drill and kill certain techniques in order to build up our pattern matching.

I don’t think they’re advocating doing hundreds of worksheets on the power rule or trig substitutions or memorizing line by line proofs. Our brains do follow formal rules when doing math, but the insight necessary to find a way to solve a problem that isn’t straightforward isn’t through application of rules, it’s through a tacit intuition that you build up by doing lots of math. There is no other way.

It’s like how everyone feels like they understand physics to a PhD level while watching the Feynman lectures, but if you were to hand them any of the problems afterwards, what seemed like such a natural stream of thought is just simply out of reach. It’s much easier to go over something and declare “this makes sense” than it is to come up with that something in the first place.

[whoisburbansky]

+1 on the drilling specific techniques. Any skill is reinforced by consistent targeted practice, and to think that math is an exception where you can break through with pure genius is just deluding yourself.

The only way I got through my undergrad math was by doing problem set after problem set until the concepts were second nature, and the courses that didn't have a sufficient breadth of exercises to drive home fundamental concepts ended up being the ones I struggled with the most.

[centimeter]

> It’s much easier to go over something and declare “this makes sense” than it is to come up with that something in the first place.

Obviously, but that’s not what we were talking about. We were comparing memorization to understanding, not inventing to learning.

If you’re good at math, you should be able to re-derive any formula or procedure quickly (up to, say, constant factors) without having to memorize it (after the derivation has been explained to you).

If you run into a problem that you can’t solve because you didn’t drill the steps hard enough, you don’t actually understand the problem. This isn’t necessarily your fault - many math courses teach by symbolic manipulation without the conceptual grounding required to actually re-derive the symbolic procedures yourself. Few students will seek that understanding on their own outside of class, in which case they’re stuck with memorization.

> the reason for Russian/Chinese dominance in certain areas of math was due to how those areas benefitted very much from rote practice.

I think this supports my point - these “certain areas” are small. There is a relative paucity of mathematical/physical innovation from China (especially per capita!). The west still dominates mathematical invention.

[0_gravitas]

Repetition to wire up thought processes and logic (why do X instead of Y) != Arbitrary memorization of trivia (X formula or Y proof)

The difference between learning X and understanding X

[Ekaros]

Training the common tools and pattern recognition for things algebra. At higher level you really need to have a repertoire of various tricks to apply to integrals and derivatives... I never properly learned these for University level.

Often the problems had some trick that you had to apply to solve it. And without knowing those certain tricks from routine finding right solution was pretty hard.

[j45]

I had a math teacher that explained math in two categories.

Either you will understand permutations and combinations almost instantly, or you will have to do tons of examples.

Likely not applicable to all kinds of math but relatable to some concepts for sure

[raxxorrax]

Calculations are based on memory but everything else not so much in my opinion. Sure, some methods for solving common problems are memorized

[bllguo]

while this sounds good on the surface, i think memorization is much more important than you give it credit. imo you absolutely should have key concepts memorized if you have any real understanding of a subject. and as you become more of a specialist, your bar for what constitutes a "key concept" should raise.

Being able to search things up is well and good, but dont you run into situations where you don't even know what to search for in the first place?

[6gvONxR4sf7o]

You can’t get intelligently conversant about a thing without having the key bits in your mind, you’re right. But you don’t need to have as much memorized as we’re were tested on when I was in school (90s-early 2000s). And you need a lot more than just memorization to reach real understanding.

Every few years I need the quadratic formula for something, and I just derive the thing instead of remembering or looking it up. I’ve essentially traded some memorization for some understanding. We’re surely going to do it wrong, so I’d err on the side of too much understanding and too little memorizing. If you have a real feel for how a thing works, it’ll stick with you longer than the date of such and such battle.

[ntsplnkv2]

> and I just derive the thing instead of remembering or looking it up.

Most people can't do this or have a very hard time learning much easier things.

I find that the crowd that talks about "pointless learning/testing/education" are often either ones that struggled mightily and were never really that smart, or they are so smart that they are above it.

[6gvONxR4sf7o]

That’s a bad example I guess. For something I struggled more with, take history. When I was in school it had a huge focus on trivia like dates, while I’m now fascinated by it, focusing on the cause and effect of things. But you can’t easily assess someone’s understanding of that side of things in a standardized manner, so it gets short shrift. Success is defined by the factors that least matter, and students, wanting to spend their efforts efficiently, will focus on the measures that define success.

As for the “most people can’t derive the quadratic formula,” you might be right about my blind spots, but I think it’s equally likely that I’m right and have the necessary point of view to see that most people can’t do it because it’s taught and tested poorly. Both explanations would equally explain it being easier for me to derive the formula than memorize it.

[ntsplnkv2]

History is certainly a better example, but dates DO matter - You can't talk about cause and effect if you don't know when things happened. Now perhaps the minutiae isn't quite as important - but what better way to learn the chronology than knowing the dates? It's fundamental to it.

> to see that most people can’t do it because it’s taught and tested poorly.

Or they just don't want to learn it. Some people don't like math. Or maybe people are lazy - they like it but can't overcome procrastination to really learn it. Or they are more focused on something else, like I was as an adolescent (computers.) There are certainly people who got sick, or went on vacation during that week of school, etc.

My point is, we always like to blame things that aren't actionable, i.e. "the system." It was just "taught poorly." There are certainly cases of that being the truth, but if you look at the time constraints and all other details, it's hard to just blame the system. How do you actually fix the system?

[6gvONxR4sf7o]

> There are certainly cases of that being the truth, but if you look at the time constraints and all other details, it's hard to just blame the system. How do you actually fix the system?

Yeah, given all the constraints, I agree completely. My complaint is that there is a giant emphasis on testing in really scalable manners that take people and dialogue out of it. So which constraint is the system (the way things are taught and tested) most sensitive to? I expect that it's teacher to student ratios, which my thesaurus says is a synonym for money, simultaneously the easiest and hardest constraint to change :-(

[Tainnor]

I guess then the point becomes - in the age of computers, what's the value in memorising the quadratic formula without understanding where it comes from?

It's much more important to know that there is a quadratic formula, or more fundamentally that every quadratic equation has 0-2 roots (exactly 2 if dealing with complex numbers) and what the different cases loom like (does the parabola touch or intersect the x-axis?). It's typically more important to be able to solve a quadratic equation through guessing, factoring or completing the square. Because these things teach you something about how mathematics works, regurgitating some formula serves no purpose, you can just ask Wolfram|Alpha instead. And even if you can't complete the square etc., I'd much rather people understood the conceptual side of it instead of remembering formulas.

[ntsplnkv2]

I don't think it has anything to do with computers. Tests have had formula sheets for decades, the quadratic formula would typically be on there.

The importance has always been "how its applied." Tons of people failed classes despite knowing the formula. It has always been about the basic application.

I don't think many math tests consisted of "write down the quadratic formula" and that's it.

[Tainnor]

The point is more that before computers, you could at least make a somewhat reasonable claim that it's useful to know the formula in case you need to solve a quadratic - even if you don't understand the formula.

But nowadays, you can just feed the equation to Wolfram|Alpha (or Sage, or Mathematica, ...), so there is no point in blindly memorising formulas.

And I don't think I've ever had a formula sheet in one of my maths exams in school...

[closeparen]

When you understand and are familiar with something you get the relevant facts in your head as a side effect. If you put them there deliberately, through the activity of memorization, that is not understanding.

[bawolff]

From what i remember of uni, most of the tests i took were not testing if you memorized a concept, but if you could apply it.

Sure the situation was still extremely contrived, but it wasn't simply regurgitating facts

[kenniskrag]

Same in university of applied science in switzerland.

For example you learn a depth first search and in the exam you have to store some information on each step to print it in the end.

You have to learn different proof schemes and use the easiest one for a given hypothesis. Sometimes you get with the wrong one in an endless recursion for example.

[Polylactic_acid]

The problem is less that students can look up answers online, its that they could hand the laptop to someone experienced and have them complete the exam for the student.

An ideal situation would be allowing students to use online resources but not to communicate with other people during the exam.

[YungSven]

My university in Germany uses this approach. We have to livestream ourselves from the side so the supervisor can see the whole desk, before each exam we have to authenticate with our Student ID so nobody else can write the test for us and we are allowed to use everything - except direct communication. This form of testing is called "Open Books" or "Open Material". Before the pandemic we were allowed to bring our books and even handwritten sheets to the exam. What I like about this is that you don't get points for just remembering things. On the other hand, this probably wouldn't be possible in subjects like economics where you have to remember a lot.

Edit: typo

[eatthoselemons]

Designing tests for open book seems to be a good way to

a) make cheating in effective, (no need to sneak in a formula sheet when everyone has it)

b) More like the real world, I always have google at my fingertips, it is just how efficiently I can use google and my existing knowledge to solve a problem

[Enginerrrd]

Thinking critically on the subject: it also prevents the problem of having someone take the test for you by feeding you information through an earpiece or chatbox on a browser tab. In places where testing fraud is rampant, this is a common form of fraud actually.

So even a well designed