[captainbland]

On "accelerate education" - it's often a bad teacher especially on higher level subjects . I've asked it to summarise research for me using the Bing GPT4 model and it would frequently come to conclusions that couldn't be corroborated in the source material, that were even contradictory across different chat sessions and even generate citation links that were totally irrelevant and incorrect, and then try to tell me that it was of a totally different subject to what it had actually pointed to.

Regular chatGPT is even more dangerous because you have no idea what it's referencing most of the time. Yet it lowers the bar to this poor information to such a degree that people will be incentivised to use it regardless.

[lijok]

That has not been my experience with ChatGPT 4. If I were to ask it about some specialized info about quantum field theory or superconductor tech, I'm sure it would send back nonsense fairly frequently. But if I ask it to explain the difference between median, mean and average, or ask for examples on an architectural pattern, it's seldom incorrect.

[captainbland]

If you prompt directly about the subject it can be fine but it veers into BS territory more often if you try to get it to synthesise information meaning it's less good with applied examples that are given to it by the user. Not always but the error potential is definitely higher.

The problem then I think is that this means that students will have to prompt "on rails" to get reliable answers. The most curious students who want to stretch their knowledge are likely to cause it to generate falsehoods which are presented confidently and convincingly.

[spion]

I'm not sure the error potential is much higher than with average teachers, but I guess it depends on the domain. For coding the error rate is lower than a typical teacher right now.

I suspect we'll soon figure out an effective way for LLMs to look-up reliable references to confirm their answers, which should improve the situation drastically. As they are now most LLMs are very barebones. An educational LLM could e.g. connect to the university textbook library and use that to verify its answers.

[captainbland]

I think the main issue today is its inability to just say "I don't know" when appropriate. A teacher can do that and sometimes that's a better answer than a fabrication. A teacher can also use less certain language when their confidence level is low as appropriate.

I think this is probably a technically solvable problem but doing so has potential 'optics'/marketing issues in terms of reducing the level of confidence it projects which a lot of people associate with competence.