[oceanhaiyang]

No one who understands ai can rely on it to help us learn. I provided one with 100 citations I wanted to standardize and it deleted 10 and made up 10 to replace them. Can’t imagine this being used to replace a textbook or even explain a textbook.

[criddell]

> explain a textbook

I've had very good luck using LLMs to do this. I paste the part of the book that I don't understand and ask questions about it.

[bigfishrunning]

But the problem is, you don't understand the passage, so therefore how will you vet the answers? Seems like hallucinations would be very very damaging in this use-case

[OtherShrezzing]

I think your mileage will vary by subject and level.

If you’re a complete novice reading a niche graduate level textbook on Tolstoy’s critique of the Russian war effort in War and Peace, you’re going to get some wild hallucinations, and you’ll have no idea how to determine fact from fiction.

If you’re reading a high school textbook about the history of pre-revolution Russia, the models will have pretty comprehensive coverage of every concept you’re likely to come across.

[palmotea]

> If you’re reading a high school textbook about the history of pre-revolution Russia, the models will have pretty comprehensive coverage of every concept you’re likely to come across.

Even in that case, it can still get its wires crossed, creating connections between those concepts that aren't true.

[lacy_tinpot]

If you can't discern what good answers look like to the questions you're asking, you're not asking the right kind of questions.

Asking the right kind of questions is a genuine skill.

It applies to every domain of life where you are at the mercy of a "professional" or at the mercy of some knowledge differential. So you need to be a good judge of whether the answers you're getting are good answers or bad answers.

[aDyslecticCrow]

> Asking the right kind of questions is a genuine skill.

A skill we cannot rely kids to have, and which i think takes years of training and learning for even adults to really acquire. (to be clear, i'm not thinking about AI prompting. I 'm thinking about assumption breaking and understanding prodding questions the learner asks themselves and seeks answers for, to build and refine their mental models of something they learn)

[lacy_tinpot]

That's absolutely not true. Kids get trained how to ask questions very quickly from a very young age. Good responses to those questions fundamentally shape the entire developmental journey for kids and extends to their academic abilities in school.

Because questions are fundamentally about knowledge differentials, which will always exist for individual human beings. We can't at any point know everything.

Know how to know what you don't know and get a good grasp of what it means to know in the first place.

Knowledge isn't absolute.

[aDyslecticCrow]

A great answer can compensate for a bad question.

A great question can compensate for a simple answer.

Kids can ask questions, but they rely on an experienced teacher to effectively answer.

Teaching someone effectively through answering questions, require the teacher through the students questions to build a model of the students model. To answer not only the question directly, but also the question that should have been asked instead.

A good end-of-chapter quiz doesn't check that a reader read the next. It asks questions whos answer rule out possible (or common) incorrect mental models the reader may have built.

A learner skilled in asking truly excellent questions, makes questions for which even a bad or simple answer rule out and refine their assumptions.

And that is a skill i doubt is ever truly mastered.

Its like the X Y. A great teacher answers X instead of Y. A great learner asks about X in the firstplace.

[squigz]

> If you can't discern what good answers look like to the questions you're asking, you're not asking the right kind of questions.

Whaaaaat? How does this work? If you're trying to learn a new topic, how are you supposed to recognize a good (and truthful) answer, whether it's from an LLM or instructor?

[lacy_tinpot]

I would argue you're doing it right now.

By being skeptical of the answers, testing the answers, corroborating with other sources, etc.

This isn't new. This is literally how we've been exploring this knowledge game for thousands of years.

[squigz]

I would argue this isn't a fair comparison. There's a big difference between a fairly open-ended discussion about a topic both parties are at least somewhat familiar with, and someone trying to learn a new subject.

[e-khadem]

Depends on the subject. For example in maths (assuming that one has a good background) you can verify the proofs yourself (and this isn't a given for highschool students).

I have also found another use for this. For example in studying modulation techniques in communication systems, I went back and forth between Monte-Carlo simulations and theoretical approximations to see how accurate each one is. And then added some more realistic error scenarios to do an end-to-end validation. In this case the LLM was used as a shortcut to write repetitive code that was verified manually, and this was complementary to the text-book, and made reading the topic more engaging, enjoyable, and comprehensive.

[0xEF]

I was in the middle of typing the same question. This is the part that worries me about Generative AI; far too many people seem to have forgotten that its prone to confabulation and telling the user what they want to hear.

[criddell]

Sure, but if the LLM tells you the jump from step 2 to 3 in a calculus problem is the use of l'hopital's rule, you should be able to figure out pretty quickly if it's a red herring or not.

[j45]

What you input along side the prompt can go a long way.

[ethmarks]

I don't follow your logic from 'When I tried it, AI hallucinated some citations' to 'AI is fundamentally incapable of explaining a textbook'. Those seem like rather different tasks to me. Plenty of humans are perfectly capable of paraphrasing and summarizing a textbook but struggle with standardizing citations.

[CamperBob2]

What is "it": what models did you try? What was your prompt? When did you try it?