[tonymet]

Instruction needs to shift to accommodate AI rather than preventing it from being used to complete assignments and tests.

Assignments and tests were always lossy, and over time more cheating crept in.

Instruction should shift to benchmarking productive output, strategic thinking and group collaboration. Similar to labs where you are tested on completing an experiment or a project with artifacts. Or an MBA program with quarterly group objectives. A major part of the group effort is dealing with collaboration and overcoming obstacles like laggards.

Hopefully people will realize how poor testing is for preparing students for the real world. the ultimate goal is preparing the students for a productive life, most commonly in commercial enterprise, but even academic pursuits require collaboration, productivity and other characteristics that were not well assessed by traditional testing and homework.

[dyauspitr]

This is nonsensical. Without a corpus of knowledge memorized and at your fingertips it’s next to impossible to build on it. Project work isn’t going to get you there. Creativity and new ideas happen when someone is deeply immersed in a space and can make connections.

[BDPW]

What would you do instead to make sure the student actually possesses the skills they are intended to have learned by the end of a program?

[saghm]

I'm not sure it's possible to force someone to learn who doesn't want to. From what I can tell, the article is basically saying that giving the students some form of agency and trust is a better way to motivate them to take it seriously than being a strict top-down disciplinarian. This fits with my experience (both from when I was a student and in my interactions with younger people as an adult), and I would expect that most people who have seriously evaluated this strategy would come to the same conclusion. It's not perfect, as some students may try to take advantage of things or will still phone it in, but the same thing happens with every other way of trying to engage with them.

[BDPW]

My experience has been the complete opposite, a bit of pressure goes a long way. There are many people who need to know X or Y and just dont have the maturity or innate motivation to do it properly. This comes from the experience of a Dutch school system so perhaps its different in other countries.

[tonymet]

A variety of performance assessments more similar to commercial pursuits.

Group projects with tangible artifacts, including finished prototypes that meet objectives. More emphasis on group projects. If AI accelerates productive development like with software, move the objectives up the ladder in complexity, or expectations.

Peer assessments and performance reviews like employment . This also helps prepare students for adult life.

If the subject matter is merchandisable, have the students operate an enterprise. My local high school has the students operate a food cart for example, and it opens to the public one weekend a month, otherwise open to students. Students are responsible for inventory, marketing, accounting, maintenance , customer service etc.

More verbal challenges . These can be operated by AI with human supervision while being recorded, with spot checks from supervisors.

Every diagnostic has a precision / recall curve and some fall through the cracks. But you have to shift your approach when old testing no longer becomes viable. Better that than to revert to the stone age of informatics.

[bonzini]

> If AI accelerates productive development like with software, move the objectives up the ladder in complexity, or expectations.

The problem is that the purpose of group projects is (besides practicing programming) to facilitate learning together, splitting tasks, discussing approaches, presenting the outcome. Doing these requires understanding what's going on, and if you just vibe code everything you don't have enough knowledge to experience the soft-skill parts of the work.

[Aurornis]

This argument has been beaten to death before AI: Ever since calculators were able to do math, students have been wondering why they need to learn how to do all of this math manually when they could get the same answer from a calculator.

The reasons become more obvious only when you get deeper into a field where the math gets too complex to get a simple answer out of a calculator. If you never learned the basic concepts, you can’t progress to the more difficult topics because you don’t have a good understanding of the foundation.

That’s why changing goals to only look at the output doesn’t work for educating kids. Now that they can have ChatGPT answer every question they might see on a middle school or even high school exam, you could conceivably get all the way through high school graduation never having learned a single thing other than how to copy and paste between the assignment and ChatGPT.

Then what happens in the real world when that student needs to learn something new? It’s obvious: They’re going to try to put the problem into ChatGPT and then give you the result back. They don’t have any foundational tools to do anything else. They haven’t even learned how to learn because there was always an easy way out. Why would anyone hire a person who can only act as an interface to ChatGPT? They won’t. They’ll use ChatGPT themselves.

My unpopular opinion is that some times hard work, memorization, doing work manually, and yes, even testing, are necessary to build up an education and thinking foundation. I don’t believe it can all be replaced by ideas about challenging students to get results and then ignoring how they arrive at the result. I’ve worked with kids enough to know that they are more resourceful about finding lazy ways to pass a test than you could ever imagine.

[FloorEgg]

What is your opinion on using an LLM to provide immediate feedback/grading at scale such that students have to muster their own answers but can check them quickly, compressing the feedback loop and allowing for more iterations?

Students still have to muster their own answers, but the LLM is used to minimize the confusion or uncertainty about the quality of the answer and the time to wait for that clarity.

My understanding is decades of research long before AI has shown the benefit of timely constructive feedback on the learning process. Why aren't all educators tripping over themselves to use LLMs to maximize access to timely constructive feedback?

[Avshalom]

Because LLMS don't provide access to timely constructive feedback.

[FloorEgg]

One interpretation of your comment is "LLM chat bots don't" or more specifically "ChatGPT doesn't", but I feel like this is a straw man and not intellectually honest answer to my question.

The real crux is not grounded in chat bots default behavior but the technology's capability: "Can LLMs provide access to timely constructive feedback in specific educational contexts?". The answer to this question is definitely yes. If you think the answer is no, then my guess is you haven't made an honest effort to try, or you just want the answer to be no and aren't interested in the truth.

[Avshalom]

>>"Can LLMs provide access to timely constructive feedback in specific educational contexts?"

"Can they" is not the same as "do they" and "specific educational contexts" is not relevant to "all educators"

Is it possible to get constructive feedback? sure, maybe. Is it possible to get a specific teacher's feedback? Not really. Is it possible to guarantee it will be productive feedback? No, especially if the student has to/gets to interact with it.

Is it likely that the reason "all educators" aren't tripping over themselves to have their students submit some number of drafts to an llm is because that's not actually a good idea? sure, probably.

[FloorEgg]

> Is it possible to get constructive feedback? sure, maybe.

Not maybe, definitely.

> it possible to get a specific teacher's feedback? Not really.

Yes actually it definitely is.

> Is it possible to guarantee it will be productive feedback? No, especially if the student has to/gets to interact with it.

Yes it is possible to guarantee it will be productive, and far more consistently productive than what teachers can achieve.

There are going to be educational contexts where LLMs can't provide productive feedback (because LLMs aren't relevant to the learning objectives). There are also many contexts where they are exceptional at producing productive feedback. Especially in grade school and qualitative undergrad courses.

I am pretty sure what's happening here is you are conflating LLMs with ChatGPT and other chat interfaces. That's a bit like conflating an internal combustion engine with a tractor, and then basing your experience with tractors on an opinion that busses can't exist. Indicated by this thing you said: "No, especially if the student has to/gets to interact with it.". It seems like you haven't considered that LLMs can have any kind of guard rails or custom instructions applied to them, can be packaged or constrained in how the user interacts with them.

An interface can allow a student to submit a draft, get static personalized pedagogically-optimized feedback tailored to the teachers criteria, learning objectives and reference material, without any way for the student to get any other output.

I find it both funny and a little irritating how confident you are that this isn't possible, because I've seen it with my own eyes used in both graduate and undergrad contexts to great success.

In a way you have answered my original query, which I am grateful for: "Why aren't all educators tripping over themselves to use LLMs to maximize access to timely constructive feedback?"

You're indicating the answer is because most educators are confidently incorrect about LLM capabilities. Plausible I guess.

[Gooblebrai]

Great comment! I'm sharing this around my circles.

[CamperBob2]

Analogies with calculators have a big problem. The calculator has no intelligence of its own. A model does. (Yes, it does. You have to be either delusional or willfully ignorant to argue otherwise at this point. Take a calculator to the IMO and see how far you get.)

So there are, or at least there will be, cases where it's actually a good idea to delegate your thinking to an AI model. Students who aren't taught to acknowledge that possibility and keep it in mind are being done a disservice, just as if they were taught to treat today's limited, early-generation LLMs as a first resort.

[mikgp]

I don’t understand the analogy you’re making, or maybe I think it’s wrong. This is the first time ive ever seen someone say you should outsource your thinking to an LLM, rather than say idea generation.

No one thinks you shouldn’t do 8 digit multiplication with a calculator, But you should understand what it’s doing under thr hood so if you say typo something you can catch when the answer is off by an order of magnitude.

But the same argument applies to AI. If you don’t understand the basics of an argument or the nature of the subject you’re investigating, you can’t tell - not even an if it’s working correctly but if it’s responding to the question you asked. If it applied the right context for your particular situation.

And I think it’s the exact same thing - whether AI is really thinking is irrelevant, students need to understand the nature of how to make arguments and validate information, before they can trust their own usage of AI.

[CamperBob2]

This is the first time ive ever seen someone say you should outsource your thinking to an LLM, rather than say idea generation.

You could think of this story[1] as an example of why it's not useful to compare AI models with pocket calculators. Here, some professional mathematicians successfully farmed out part of their thought process to an AI model.

It doesn't mean they shut their own brains down, or that they are suggesting that students do so, only that they allowed for the possibility that the model might be able to see something they didn't see themselves, or make a connection that no one had considered before.

1: https://www.scientificamerican.com/article/ai-just-solved-an...

[Avshalom]

>>It doesn't mean they shut their own brains down, or that they are suggesting that students do so

It doesn't not mean that either.

[randsorex]

I suspect it is all futile without resurrecting the old idea of being "learned" as in learn-ed.

"Learned" didn't really mean what we mean today by being well educated or smart. You can't use AI to cheat and become "learned". AI can find the books to read but you still have to read them and understand the ideas.

There was connotation of breadth as opposed to depth with being "learned".

I think we also have to forget about "the real world". Being "learned" automatically is going to inherit dealing with "the real world" because the real world is always changing and that is exactly why breadth should be the focus going forward more than the depth of the research university model.

Of course, in a society so dominated by credentialism, credentialed people are going to hate AI because it will obviously let anyone cheat at the credential they put so much time and effort into. This doesn't need to be dressed up in some "think of the children" argument.

Claude to me is the greatest thing since sliced bread that increases my "learnedness" every single day but I also am a drop out that invested basically nothing in being a credentialed person.