Hacker News new | ask | show | jobs
by defnotahuman 1262 days ago
The problem is that calculators are correct 100% of the time, ChatGPT is not. What makes it more difficult is that it's often really hard to tell when ChatGPT is correct or not.
4 comments

The problem is, people assume the tools they use are correct 100% of the time.

An early example: http://files.righto.com/calculator/sinclair_scientific_simul... (see Bugs and limitations).

More contemporary (I think): https://apcentral.collegeboard.org/courses/resources/example...

Granted though, ChatGPT is confidently and convincingly wrong more often. I expect that will improve, but in the meantime I think it's an opportunity to question and evaluate the results rather than ban outright. That's a sticking plaster that hopefully needn't stay on for long.

For whom is that a problem, in this case? If you have a hallucinating computer do your test and it doesn’t work out for you, that’s on you.
I think it boils down to the (intellectual) effort needed to synthesise your thoughts, which is a skill that leads to deeper understanding.

I'd say ChatGPT is a bit better as a tool than copying from a colleague, just because you still have to double check for correctness. However, does it rob students of that opportunity to think deeply and therefore learn? I'm not sure to what extent, but I'd imagine that there's an impact to _some_ extent.

ChatGPT is also going to get better, and will eventually be pretty accurate (or, at least as accurate as a student of a subject may be). My stance is that in school, you should be learning to think instead of memorising. If the goal is to memorise, however, then use ChatGPT isn't that harmful (you'll still fail for memorising the wrong information).

Overall I'd treat ChatGPT as a tool, and give students the facts: first gain a deep understanding of a subject so you can learn to verify, then use the tool later in life.

I'm also mindful of the fact that if an AI is not guaranteed to be correct, which is the case right now, then students now have an incentive to _understand_ the subject so they can verify, which might actually be a net good?

This same argument is valid for people. That does not mean people are useless, for example as assistent. Maybe we should just use that knowledge and still try get the best out of it.
Neither is Wikipedia. While some vandalism might be obvious, more nefarious edits might not be. It's still an extremely useful tool.