Hacker News new | ask | show | jobs
by aeve890 31 days ago
>It definitely learns things.

By the very way this technology works they can't learn anything after training. What you think is "learning" it's just a session log written back to the context when you resume the session.

2 comments

It's literally called in-context learning. The fact that it doesn't go into weights and is retained only for a session doesn't mean no learning occurs.

Also agentic systems may choose or be instructed to retain some information between sessions in files and/or databases which is also a form of learning.

There are experiments with retaining session information in weights in some form of lora but there's no consensus if it's even desirable. There's a value in being able to start from a clean slate.

And what is written when you wake up from sleeping?
It's absolutely not the same. If you think llms and brains works the same way you clearly don't know how either works.

For a LLM learning what you wrote your last session would be update the weights with the new relationships and factual knowledge created in the session. That doesn't happen. The weights are static and fixed after training. There's no online training in the transformer architecture or any variant. If the weights don't update, the network doesn't learn. Period.