Hacker News new | ask | show | jobs
by kingkongjaffa 657 days ago
The main issue with LLM powered learning is you need to have at least a surface level understanding of the topic to recognise if the output is wrong.

That requires some basic knowledge, a specific mental model for interacting with LLM’s, and a modicum of good taste in code to be able to correct for the code smells that LLM’s fall victim to.

You can never let the LLM take the wheel, you need to be the driver knowing what you want. This works if for example you know how to something in language x but need it in language Y, but it doesn’t work if you have never been able to solve the problem before.

For something like leetcode however which is far from novel, there are that many correct answers in the training data that the risk is low.

6 comments

I am so glad I got my basics in before this AI era, I feel sorry for the folks who are just learning to code now. AI can do all the basic tasks and frameworks so it will probably feel so futile for them to try to learn it all by themselves.
This.

I don't know why, but I find it easy to imagine a future where real programmers who understand both the craft and the art from its lowest fundamentals to the highest levels of abstraction are the equivalent of https://en.wikipedia.org/wiki/Low-background_steel.

I'm seeing this play out among young people I know who were trying to improve their English writing skills. Everything they post now in any formal or semi-formal context is via ChatGPT. Their self-confidence has taken a nosedive.
I feel Programming is more about learning how to think and break problems down in smaller parts. That skill should always be useful
i think textbooks are still good
It's like the skill needed for calculator usage. You need to be able to quickly estimate what rough order of magnitude, first and last couple of digits, to check that the result "looks right". Using a calculator to do 100 quick sums is a different skillset to doing 10 by hand.
Very very good point and often understated imo.

Yes, LLMs are okay-ish at javascript and python, but if I ask it to generate me some VHDL, it often just produces plain garbage. And without a basic understanding of hardware design you wouldn't even know what to look out for. It's just not possible to learn anything from it.

> you need to have at least a surface level understanding of the topic to recognise if the output is wrong.

Often overlooked point. The only reason I even consider LLMs a useful tool for my work is because for the things I request from them I do have the prerequisites to validate their output.

The classic "How Many R's in 'Strawberry'?" LLM test is amusing to students...

Still useless for trying to understand subtle complex concepts not previously well-defined in the data sets.

The numerous slop articles posted on YC that obviously were partially written with LLM help often makes the alleged authors sound like they had a stroke. lol =3

Agreed. But LLMs are pretty good at answering questions and helping students understand concepts from fixed datasets ie. courses. That is what we're trying to achieve at Edmigo
Passively cramming information will not retain a lesson more than 3 months. Additionally, only 1/17 students have the discipline to succeed with self-directed studies.

I remain skeptical the current well-formatted LLM nonsense will help students, and have concerns it may cause hindrance with long-term information recall.

We shall soon see I guess... Best of luck, =3

I totally agree that passive learning is not useful. So Edmigo uses a hands-on learning approach where students solve problems on their own alongside an AI guide
Curious where you found the 1/17 statistic?
It is an internal metric from a College that offers kids GED/remedial-high-school-level math prep classes for remote students. There are various reasons a student may be unable to physically attend classes, and remote 1-on-1 instruction is not in the budgets. The rate for curriculum completion is a very low 1:17 for self-directed study.

The learning outcomes for students are often terrible for this path, but perhaps good if you charge for the tutorial service I guess. Attrition rates are not something institutions proudly promote in public.

YC seems to down-vote anything that deals with a data-driven reality. lol =3

https://www.youtube.com/watch?v=GYh7smM6YpM

Interesting points. but not very student can afford or has access to group studies or human tutors right?
LLMs kind of are, but imo it never meets up for a
Very valid points. We've controlled these issues by providing content which we created ourselves and running other pipelines to validate the tutor's answers