[squidbeak]

They learn between model iterations. You're right, it isn't the same thing as Junior developers' competence improving with experience - the current model's weaknesses are locked in. But it does mean that much of the Junior level thinking and mistakes will be outgrown by successor models.

[tremon]

But they don't retain anything from your on-the-job training. The next model iteration is yet another junior fresh out of college, and knows nothing about the painful training procedures its predecessor put you through.

[codebje]

Yes... but the next session with the same model is yet another junior fresh out of college that knows nothing about the painful lessons the last session put you through ten minutes ago, either.

[ACCount37]

Skill issue?

Nothing prevents an LLM agent from writing a bunch of "notes to self" and using that. And the next model from picking those notes up and using them. Coding agents already do some of that natively.

Hell, we might eventually get an LLM to say "wow the old AI was an incompetent idiot" after reviewing all the notes and session logs. That's how we know we reached human parity!

[codebje]

The context window limit prevents it, for one.

[ACCount37]

Only if you are incapable of fitting both the task and task-relevant data into it. And 1M contexts are mainstream by now.

Context size is a capacity limit, not a showstopper.

[fc417fc802]

Surely you just copy the prompt over and it immediately knows all the same on the job stuff that the previous model did.

[hibgymnb]

The point is the current model also knows nothing about the “on the job stuff”.

It’s extremely difficult(impossible?) to include every bit of relevant domain knowledge into “the prompt”