I've said it in another thread, but if you have to ban a tool to preserve the value of some course or examination then it's likely they have little to no value in the real world.
In most cases the courses are fine, but the examination probably isn't testing anything useful if it can be cheated with tools like ChatGPT. For example any maths test which only tests students on problems which are easily solvable by a calculator isn't testing any skill of real world use. But this doesn't mean maths itself isn't valuable, just that the test isn't.
In some cases like programmers and writers, ChatGPT might actually be eroding the value the current curriculum. For example, I'm not sure if there's much value in learning things like basic SQL queries anymore which was something I had to learn when studying computer science. ChatGPT is great at writing SQL queries if you're able to tell it exactly what you need.
> I'm not sure if there's much value in learning things like basic SQL queries anymore which was something I had to learn when studying computer science.
I really disagree, a major function of education is to teach concepts before introducing tools. I took a physics class in undergrad that required a graphing calculator, does that mean I wasted years learning all of the mathematical concepts that could be performed by a graphing calculator?
Having an understanding of what's going on is so important, especially when something is broken. If we encourage students to start with AI and not learn the fundamentals themselves, they won't wind up learning how to identify root cause when the tool causes an issue (and ChatGPT is by no means perfect).
Obviously it's nonsensical to say students should have access to all tools. In the real world every single idea and fact is easily look up able, but by teaching students to derive known facts gives them the skills to eventually derive unknown facts in the future. But most people have a super neoliberal view of education and pretty much want every American student to be an idiot who just can't think
If a student can look up anything on ChatGPT it could very well inhibit their ability to develop critical thinking skills. They could just copy the output of ChatGPT rather than actually engaging with a text like Of Mice and Men, for example, and considering what it's trying to say about the American dream or interpersonal connections. Lots of learning exercises are relatively meaningless in isolation but are a part of a greater skill development journey. It's like just taking the answers from the back of the book rather than working through exercises. Getting the answer isn't the point. The point is actually doing the work so you develop skills that can be used beyond your current homework assignment or exam.
When I was in high school we 'barely' had search engines. I graduated in 98, I'm not even sure google was around then. I've never seen them try to get rid of google which has just as much ability to inhibit critical thinking skills. I mean imagine having to learn and remember code constructs instead of being able to search online. Imagine if all your docs were books, and that's it. Sure you'd need to retain more, but would you be more performant because of it? We're moving to a society where ai-augmentation will be the norm, not the exception and we need to start training on that premise so that we can perform to our best when we have more info at our fingertips.
This feels like it could be solved partly, and more effectively, by a step wise show your work process. So if it’s Of Mice and Men, it could be a reading journal, then important points, then an outline, then an essay.
If they successfully use ChatGTP to do this, they likely have engaged with the text.
> For example, I'm not sure if there's much value in learning things like basic SQL queries anymore which was something I had to learn when studying computer science. ChatGPT is great at writing SQL queries if you're able to tell it exactly what you need.
Using ChatGPT to generate something that you can't independently verify is a terrible idea. Most of the time it'll be fine, but when it's not you need to be able to recognize the flaws. Hopefully you at least test it before pushing, but if you don't know how to write the query you also don't know its failure modes, so your tests will likely miss important edge cases.
AI tools are great for speeding up processes you can do by hand, but because they're all statistics you can't count on them to do it alone: some percentage of the time they will inevitably be wrong.
EDIT: Note that this stands in contrast to a wholly deterministic tool like a calculator, which in most circumstances doesn't need to be second guessed.
> Using ChatGPT to generate something that you can't independently verify is a terrible idea
It depends. A calculator can calculate things I'd struggle to independently verify with a pen and paper.
I think I wasn't clear enough and you might be taking what I said to it's extreme. I'm not suggesting there is zero value in knowing SQL, I'm saying the value of learning it is now much lower than when I learnt it. While it might be helpful to have a good understanding of SQL to verify a query yourself (especially where performance is a concern), in most cases this likely isn't necessary. For your average select query with a couple of joins and a where clause ChatGPT can generate that and you can verify it at least works by running it.
But maybe SQL isn't the best example. It just came to my mind because I was writing a spatial query for MariaDb recently and I couldn't get the results I wanted after a few Google searches, but ChatGPT got it right first time. It seems to me that query languages have a lot of properties which make ChatGPT very good at writing them, but there are probably other skills which ChatGPT more clearly erodes the value of.
I don’t entirely agree with this. My math education relied way to heavily on mechanical substitute-and-compute, but there’s definitely value in making sure that children can actually do and understand what computations are actually happening.
I would imagine writing is similar - the endless essays full of bullshit I pumped out in high school were obviously pointless. But being forced to write much earlier in school was super foundational for having the skill in the first place.
As to the SQL example, I also think it’s flipped - learning the basic sql queries is probably the most valuable as it’s a foundation.
> For example any maths test which only tests students on problems which are easily solvable by a calculator isn't testing any skill of real world use.
True in university, but memorizing multiplication tables and learning the mechanics of long division and multiplication are important parts or math education for kids. Most of school is not strictly for "real world" skills anyway (even if doing basic math unaided is almost as close as we come in school to teaching a real world skill).
Same goes for say, basic reading comprehension and summarization.
This would replace the skill of SQL with the skill of being able to ask specific questions which is a valuable skill, also when collaborating with humans.
At the same time it is a completely different mode to operate. Writing and editing SQL while you are trying to solve a problem will lead you to a different way of thinking, iterating, maybe even induce fewer context switches.
Using ChatGPT can be similar to delegating those type of tasks to a colleague. There are upsides and downsides to this but never underestimate the benefits of strengthening your own abilities.
While I see your point, it is not hard to see the implications of your argument. Should students be allowed to use calculators in basic math? Don't you think they should learn arithmetic and be tested on that? What about reading? Why learn how to read when you can just use a device to read text to you?
When you first learn it is generally important to learn the basics without tools. In your example knowing the times takes, at least for single digits is important... But can be fine with a calculator.
When you first learn SQL knowing sums simple queries is important. Maybe they'll pick it up after asking chatgpt, stack overflow, or Google, but one shouldn't have to do a search everytime they need to do a basic thing
Not sure I entirely agree. Imagine there's a trainable skill like writing, and GPT can write at low-intermediate level, but can't at high level. How do you test students before they completed intermediate level?
I agree about handwriting being pointless, but simply doing writing exercises in the classroom should be sufficient to get students past the ChatGPT level. Right?
Assuming the teacher is doing their job, it should be really easy to make sure nobody is cheating using machine learning.
I mean, something like chatGPT (that will refuse to invent things) is perfect for student companion. It's endlessly patient and willing to explain every word you don't understand. No teacher can do that to this degree. If i were a teacher i d want to offload work to it.
ChatGPT is just a tool, like a calculator. Calculators are banned in certain circumstances and required in others. At first they were probably seen as cheating, but then the system realized they could be used to enhance what kids learn. Why take away a tool that could be used everywhere in life? That's not how progress should work. The system should evolve to embrace such tools, but the obvious problem is that it just won't happen quickly enough, and in the short term they will be used for illegitimate purposes, and so we just ban.
Did you even read the article? You completely side stepped the argument they are making:
> While the tool may be able to provide quick and easy answers to questions, it does not build critical-thinking and problem-solving skills, which are essential for academic and lifelong success
I think we really need to get on integrating technology like this into a curriculum which does build critical thinking and problem solving skills, since these kids are already inundated with AI generated content and it seems certain to increase several fold over the next few years.
Ok, this is why I think it's wrong: ChatGPT absolutely does teach us critical thinking and problem solving skills. How to best leverage ChatGPT is absolutely a problem that needs to be solved, and it varies wildly on the use case. Critical thinking absolutely does need to be applied to what it produces. All it can do without critical thinking and problem solving skills is create commoditized responses, the same thing it produces for everyone else. That has no value anymore. HOW you leverage is insanely important.
The problem is that calculators are correct 100% of the time, ChatGPT is not. What makes it more difficult is that it's often really hard to tell when ChatGPT is correct or not.
Granted though, ChatGPT is confidently and convincingly wrong more often. I expect that will improve, but in the meantime I think it's an opportunity to question and evaluate the results rather than ban outright. That's a sticking plaster that hopefully needn't stay on for long.
I think it boils down to the (intellectual) effort needed to synthesise your thoughts, which is a skill that leads to deeper understanding.
I'd say ChatGPT is a bit better as a tool than copying from a colleague, just because you still have to double check for correctness. However, does it rob students of that opportunity to think deeply and therefore learn? I'm not sure to what extent, but I'd imagine that there's an impact to _some_ extent.
ChatGPT is also going to get better, and will eventually be pretty accurate (or, at least as accurate as a student of a subject may be). My stance is that in school, you should be learning to think instead of memorising. If the goal is to memorise, however, then use ChatGPT isn't that harmful (you'll still fail for memorising the wrong information).
Overall I'd treat ChatGPT as a tool, and give students the facts: first gain a deep understanding of a subject so you can learn to verify, then use the tool later in life.
I'm also mindful of the fact that if an AI is not guaranteed to be correct, which is the case right now, then students now have an incentive to _understand_ the subject so they can verify, which might actually be a net good?
This same argument is valid for people.
That does not mean people are useless, for example as assistent.
Maybe we should just use that knowledge and still try get the best out of it.
I see ChatGPT more akin to Wikipedia and Google than a calculator. We don't consider Wikipedia or Google as much of a tool as they are a knowledge reservoir of truthiness.
It's absolutely a knowledge base. The "training data" is its raw database which is then preprocessed to speed up the query engine. Then it does some additional processing to return amalgamated results from that database when presented with a query, which for some reason they call a "prompt." It's clever, but when you view the training data as part of the program source code, which it absolutely is, the information entropy of its output is miniscule.
- Google: How do I ...
- visit stack overflow
- copy/paste
- code generated.
There's more steps, but how is this different? Chat GPT though will go the extra mile and actually EXPLAIN what each bit does usually. It's not always accurate but neither is stackoverflow.
The only knowledge this system knows is whatever it can scrape and regurgitate that appears truthful. A whole bunch of positively true, but also many cases of falsely positive.
This system can’t prove its own work (at least not yet) before publishing the results. It just publishes, which is similar to Wikipedia and Google.
Some of the Calculator analogy comments downstream are inaccurate.
300,000 students in the United States compete in the AMC 8/10/12 exams every year. Calculators have always been banned. The official policy[1] states sternly in uppercase -
---
AMC 8
The only materials that students are allowed to have on themselves during the competition are writing utensils, blank scratch paper, rulers, and erasers. NO CALCULATORS OR PHONES AND SIMILAR ELECTRONIC DEVICES OF ANY KIND ARE ALLOWED. No questions require the use of a calculator.
AMC 10/12 - AIME
The only materials that students are allowed to have on themselves during the competition are writing utensils, blank scratch paper, rulers, compasses, and erasers. NO CALCULATORS OR PHONES AND SIMILAR ELECTRONIC DEVICES OF ANY KIND ARE ALLOWED. No questions require the use of a calculator.
--
Mathcounts[2] is kinder, but only a little bit - "Calculators are not permitted in the Sprint and Countdown Rounds, but they are permitted in the Target, Team and Tiebreaker Rounds". There are 500+ chapter rounds every year, plus 56 state rounds, & each school sends upto 12 contestants, so again a very large number of students compete in these exams with no calculator.
Note when they say "No questions require the use of a calculator" - that's quite a dubious claim. For eg. the question which is bigger 9^10 or 10^9 ? With a calculator it becomes trivial. Without, you take logs to base 10 and do some simple arithmetic by hand. For most trig problem sets, you'd have to know the standard sines & cosines. With a calculator that becomes moot.
I wonder if the script can be flipped to both encourage the use of ChatGPT and scrutinize its output among students. I imagine that analyzing the results of ChatGPT output for something like a routine essay prompt requires a higher degree of precision and subject-matter expertise than writing the essay itself.
I'm sure they're concerned about essay writing, but my kid has been using to help with physics and math homework. Not for basic calculator functions, but it's surprisingly good at giving formulas for complex problems. You can ask things like how to calculate the angle in degrees of fired projectile given initial velocity and range and it will give a formula. Then you can rephrase for how to solve for velocity given range and angle.
If I recall anything about my days in the DOE, a motivated student won’t have any problems getting their hands on a banned ChatGPT on a school network.
My question is, did you learn the truth from ChatGPT? It seems like it will always give plausible sounding answer, that may or may not be correct. For example I asked it today to enumerate the rights given to all christians as part of the priesthood of believers according to Luther. It enumerate a couple of plausible sounding rights. Unfortunately none of them were correct.
> It tends to land in the right ballpark and capture the tone of a thing.
One weird trick is to use this text to search the web. Because it is already in the ballpark it will be a very good query. The model would bullshit "The height of Everest is 8,230 m", you take this string and search it, find "height of Everest is 8,849 m" -> correct the original response.
Now the problem is, how do you decide what information to trust in search?
AI needs a big push to index and do consistency checks for all facts. Let's have the model write a billion wiki entries and knowledge base concepts. Check for support, consistency, competing explanations - everything should be in there, we don't decide what is true. Then a language model can say when a topic is controversial, or when it doesn't know something.
Of course the goal is to find the truth, but we know it is going to be tricky. In the meantime we can have models that know when they don't know, or when the information is not certain.
I definitely agree with this, but what I have found it useful for is for discovering 'unknown unknowns' that I can then research in more depth.
It's too bad that it produces such plausible-sounding answers while being subtly wrong. But it can certain help people to expand their knowledge. I don't think it's a replacement for a comprehensive education on a topic, but can help people get closer.
I am learning a JS game dev platform and ChatGPT has been a GREAT help, even though it’s only right 90% of the time.
It cooler if it were an “auto-answer” tool for Stack Overflow. That way I could still get instant answers, and the community could asynchronously validate the answers.
If you’re learning something, glow do you know if it’s correct or wrong. I can see it as a tool to remind you (like damn I forgot how to reverse a linked list) but not on a topic that you wouldn’t be able to remember or not since you didn’t know it in the first place.
ChatGPT is a chatbot, not Google. It just produces answers that are plausible-sounding. ChatGPT is not doing any searches or data lookups for you in the background. Unfortunately, you may have learned total garbage.
This is just going to mean students use it from home or from their phones. And the issue is that you can't prove ChatGPT has been used, so cheating is that simple.
this story made me realize we’re amazed by chatgpt’s ability to answer our questions. But how about chatgpt’s ability to evaluate our answer to its questions ? Never seen prompts of chatgpt acting like a teacher. Wonder how that would look.
Typically, yes, a student in fourth grade learning multiplication isn't allowed to use a calculator. We tend to assume that you need to be able to do something by hand first so you have an intuition for what a right answer looks like. Only then can you use the machine to speed up the process.
In most cases the courses are fine, but the examination probably isn't testing anything useful if it can be cheated with tools like ChatGPT. For example any maths test which only tests students on problems which are easily solvable by a calculator isn't testing any skill of real world use. But this doesn't mean maths itself isn't valuable, just that the test isn't.
In some cases like programmers and writers, ChatGPT might actually be eroding the value the current curriculum. For example, I'm not sure if there's much value in learning things like basic SQL queries anymore which was something I had to learn when studying computer science. ChatGPT is great at writing SQL queries if you're able to tell it exactly what you need.