[spion]

I'm not sure the error potential is much higher than with average teachers, but I guess it depends on the domain. For coding the error rate is lower than a typical teacher right now.

I suspect we'll soon figure out an effective way for LLMs to look-up reliable references to confirm their answers, which should improve the situation drastically. As they are now most LLMs are very barebones. An educational LLM could e.g. connect to the university textbook library and use that to verify its answers.

[captainbland]

I think the main issue today is its inability to just say "I don't know" when appropriate. A teacher can do that and sometimes that's a better answer than a fabrication. A teacher can also use less certain language when their confidence level is low as appropriate.

I think this is probably a technically solvable problem but doing so has potential 'optics'/marketing issues in terms of reducing the level of confidence it projects which a lot of people associate with competence.