[iepathos]

The article, the teacher, and the general academic community skips the hard question when it comes to AI and that's whether these exams are testing knowledge that is still worth internalizing in the same way?

Academia has a long history of lagging behind acceptance of new cognitive tools where they claim to want to defend the students, but instead defend the assignments of the past at the expense of the students. Calculators were treated as threats to learning, even though they ultimately freed students to focus on higher-level math and provably improved their abilities across many different studies. Internet sources were dismissed as less legitimate than books, as if “published in an outdated book from the 70s” magically made it more trustworthy than the most scrutinized reference sources online.

It is not clear from the article exactly how much of this course falls into that category, but if the answers can be produced trivially with a prompt and chatgpt, then maybe memorizing that material is no longer the right educational target. Academia desperately needs to redesign itself around AI as a cognitive tool students should be trained to leverage. If a question is trivially answered by a prompt with it, then you need harder questions that actually require students to push beyond that. Simply removing AI from the equation, calling it cheating, and pretending that it isn't an ever-present asset people are expected to leverage in real life is naive and just repeats the mistakes of the past.

[versteegen]

> whether these exams are testing knowledge that is still worth internalizing ... It is not clear from the article exactly how much of this course falls into that category

It's very clear from the (excellent) article linked by dang [1] what the exams required:

> This year, the economist decided that both the midterm and the final exams for his course would be of the take-home, closed-book type (there is a certain tradition of this at Ivy League schools). “It’s a very nice kind of exam, because as you’re giving students practically unlimited time to complete it, it lets you make it harder than normal, to see how far they can go.” In this case, Serrano changed some of the model assumptions they had seen in class, and asked students to demonstrate whether certain statements were true or false under the new assumptions.

[1] https://english.elpais.com/education/2026-06-28/ai-fraud-at-...

[episteme]

I don't think there's anything wrong with asking for things that can be prompted. You still need to understand the why behind things, being able to reason about them and choose between options. How will you teach this level of understanding or certify them without exams?

Of course, not all testing is good, but the written exam has survived and proven useful despite the internet age, I'm not sure an even better search engine really changes that.

[mattas]

Why bother learning anything when you can just use AI?

[MarsIronPI]

So you can call out the AI's mistakes. Everyone seems to be saying that AI is really good, except at their particular field. Make of that what you will.

[asdfasvea]

But here's the thing, AI will be the source of truth soon. For many it already is.

The dumber we (humanity in general) get, the smarter AI appears to the average person. You'll be a heretic or just deemed crazy if you call BS on something AI says in 50 years--regardless.

[MarsIronPI]

> But here's the thing, AI will be the source of truth soon. For many it already is.

When that happens, the bridges will start collapsing and the generators will stop working. So I'm confident that when we get to that point people will come to their senses, at the latest.

[lubujackson]

You need to know what to ask and what the answer means. Also, using proper terminology greatly increases LLM accuracy to situate the response in the correct domain.

Ask an LLM to "clean up this code applying SOLID principles" is much more prescriptive than "clean up this code".

[antisthenes]

Exactly.

AI can just give you the details like an encyclopedia article. You still have to know at a high level how something works and which article to look for.

AI is just a more forgiving way to ask something, compared to the searches of the past (e.g. searching a scientific journal database). And (for now at least), AI is a good way to avoid the ad-ridden web.

[parsimo2010]

I have a few thoughts related to this, and maybe I can get them out and ties them back together at the end.

1a. Yes, college isn’t right for everyone, and testing the traditional way with paper and pencil certainly disadvantages some students who would be star performers in a real world setting but are not a good fit for college classes. Think “Good Will Hunting” type people. They do exist.

1b. However, there are certainly more people who only imagine themselves as Will Hunting type people and that they are just too smart for college, but the reality is that they are dumb, or didn’t learn the material. For every person who fails a test because they’re a genius who is a bad fit in the system, there are at least 10 idiots who imagine themselves geniuses, and they would have passed the class if only that PhD professor with all his book smarts had actually written the right kind of exam. Traditional schooling and testing doesn’t work for the extreme upper tail of intelligence, but it exists because it did quite well at educating and sorting the masses to support the Industrial Revolution.

1c. If college were only about the knowledge, you could learn most of it with internet access and a library card for much cheaper. The vast majority of people are in college for the credential, and the institution has to protect the signal of the credential.

2a. Most college assignments and exams are not a good reflection of full time employment. College credentials serve mainly as a networking aid and a signal to employers that you are compliant and competent enough to follow a professor’s instructions, and will likely be a compliant and competent employee, although the instructions might be different. If it is the AI who is the one who followed the professor’s instructions you water the signal down, and employers don’t want that. It is irrelevant that a student can copy and paste things into ChatGPT, and that ChatGPT can get the answer right on this test. That isn’t what college is supposed to signal.

2b. A problem is that I literally can’t write a test in most subjects now that I would expect a student to complete that can’t be completed by ChatGPT better and faster. I teach undergraduate math and a while ago we thought that since GPT-4o could get a C in calculus that we’d just raise the standard. Now Fable and GPT-5.5 can cruise to an A in literally every math course in our catalog, and they can also catch every tiny issue in an exam written by a human. But I have to teach these undergraduate subjects so that some students can go on to PhD studies so they can contribute to the field. If we just stop teaching undergraduate subjects then PhD production and novel research grinds to a halt and only a few fields will progress where an AI is capable of self improvement.

2c. I’ve seen that my best students know how to do do something by hand and use a computer to complement/increase their capabilities, not to cover over entire gaps. When you have literally zero skill in an area, you can’t spot when you got a totally bad output from the AI (these days usually because of a lack of context or bad prompting because the student didn’t understand the material, not because the AI wasn’t capable). Somehow I have to incentivize students to learn the material on their own, so that they can be a better user of AI in the future. And a really effective way to do that is a graded test in which AI is not available to help them.

So to try to tie it together, is that there is still some value to a college degree, at least until something better comes along. But that college degree is only useful when it is a signal about the person and not some other tool. And although AI is getting very capable, somehow we have to teach the lower stuff to build up to the higher stuff, so we do need to restrict the use of AI in some educational settings so that we can build a good foundation for future learning.

As to the point about old paper sources being considered more reliable than an internet site, I agree. I am of the generation that wasn’t allowed to cite Wikipedia and it frustrated me. We’ll eventually figure out how to permit proper AI use, much like how many professors now allow you to use Wikipedia to start researching a topic.

[lubujackson]

I think the reality is that you should focus on and serve the 2c group (best students who use AI to fill gaps or speed up learning). We have 10s of thousands (or more) students taking advanced math every year. Only a slice of those pursue PhDs and even fewer contribute anything meaningful.

This is how it has been for years and AI doesn't really stunt that as long as students actually care about learning the material. Those kids will be fine. The kids that are fulfilling a prereq will be fine, for the most part, but will walk on crutches their whole lives (use AI) which might be a reasonable choice.

What we lose is the signal of which kids are interested in learning and which kids aren't. Grades and effort used to be the signal, now I think it has to be more about engagement" who asks questions, who comes in with strange ideas, who actually is struggling at times (actually grappling with the material) or learning things outside of the assignments.

Big picture, I think teachers should focus on teaching the learners and trying to motivate more students to join them. That has been the real job all along, AI just lays it bare.

[threatofrain]

It's worth noting that even though you say people could just go online and start reading, college may be the last time most people ever read a textbook ever again. There's something tough about knowing that you want to do something but then somehow your daily instincts fight you.

It's also why you can't just throw kids in front of Khan Academy.

[parsimo2010]

> Somehow I have to incentivize students to learn the material on their own... And a really effective way to do that is a graded test ...

I know. The desire to get a degree or the "threat" of wasted tuition and disappointed family members is really what gets most students to crack a book. Most people aren't self-motivated to learn. If people were self-motivated to learn the material, as opposed to getting a credential or some other reason for being in college, I wouldn't need to worry about cheaters.

[palmotea]

> Calculators were treated as threats to learning, even though they ultimately freed students to focus on higher-level math and provably improved their abilities across many different studies.

Oh fuck that bullshit. I was a dumbass to believed shit like that in school, and it didn't fucking free me. What happened was I was always calculator-dependent, which made higher-level math harder, because I was always distracting myself by operating the goddamn thing, and I never developed a very good intuition for arithmetic.

I fucking hate calculators in math classes.