[toss1]

>>What makes LLMs different?

Good question!

Writing or calculators likely do reduce our ability memorize vast amounts of text or do arithmetic in our heads; but to write or do math with writing and calculation, we still must fully load those intermediate facts into our brain and fully understand what was previously written down or calculated to wield and wrangle it into a new piece of work.

In contrast, LLMs (unless used with great care as only one research input) can produce a fully written answer without ever really requiring the 'author' to fully load the details of the work into their brain. LLMs basically reduc ethe task to editing not writing. As editing is not the same as writing, so it is no surprise this study shows an serious inability to remember quotes from the "written" piece.

Perhaps it is similar to learning a new language wherein we tend to be much sooner able to read the new language at a higher complexity than write or speak it?

[cabacon]

I have a kid in high school who uses LLMs to get feedback on essays he has written. It will come back with responses like "you failed to give good evidence to support your point that [X]", or "most readers prefer you to include more elaboration on how you changed subject from [Y] to [Z]".

You (and another respondent) both cite the case where someone unthinkingly generates a large swath of text using the LLM, but that's not the only modality for incorporating LLMs into writing. I'm with you both on your examples, fwiw, I just think that only thinking about that way of using LLMs for writing is putting on blinders to the productive ways that they can be used.

It feels to me like people are reacting to the idea that we haven't figured out how to work it into our pedagogy, and that their existence hurts certain ways we've become accustomed to measuring people having learned what we intended them to learn. There's certainly a lot of societal adaptation that should put guardrails around their utility to us, but when I see "They will make us dumb!" it just sets of a contrarian reaction in me.

[toss1]

YES, fully agree, and your kid is definitely doing it right!

I've also found LLMs to be very helpful in proofreading to find inconsistencies, missing items, stray edits, etc..

Here's one variant of prompt I've used with ChatGPT-4o that worked well:

"Focus particularly on inconsistencies and editing errors (stray words or characters, etc.). Be exact and do not include compliments to the author. Please ignore apparent duplicate listings of part numbers and dimensions which are under the illustrations, and single stray or inconsistent spaces."

I'm quite sure that LLMs used in the right way can be amazing for teaching, and I've used them to learn quite a few things. In fact, it seems this is one of the strengths of LLMs — they are not so good at 'reasoning' about unusual content at the edge of a field of knowledge, but are fantastic for compiling info that is commonly used by humans but not yet familiar to a particular human.

And yes, the "They will make us dumb!" response kind of depends if you are starting out dumb — just as a hammer will make you smarter if you use it to pound nails to build a house and a school, but dumber if you hit yourself in the head with it...