[sureMan6]

Might bother you to use anthropomorphic terminology like smart and learning but they are capable of producing work that traditionally required human intelligence and the whole point of gpt 3 was the ability to "learn", you can give it an example of an invented brand new coding language and it can write working code in that language

[kmacdough]

Context is not the same as learning. It's easy to conflate because they're tightly coupled in our brains.

The underlying structure and tuning of the LLM are entirely unchanged by context. It merely affects the attention and activation of the network. The LLM will not be able to work with this hypothetical new language unless it is in context. This does not fit the computational meaning of learning.

Smart is not a well defined term. Nor is it's general idea formally understood. Use it freely, but you won't be saying anything meaningful unless you define your usage.

[DiogenesKynikos]

The LLM is the model + context. The output depends on both.

You're making an artificial distinction. The LLM sees a new programming language for the first time, and can immediately code in it. That's learning by any reasonable definition.

If you go past the context window, it forgets, which is a limitation of current LLMs. But as long as it learned how to code in the new language within its context window, it has gained that new ability.

[daveguy]

The context is the input to the LLM model. It seems like you need to study up on how LLMs work, instead of spouting hype.

[DiogenesKynikos]

It sounds like you need to be less condescending and engage with my actual comment.

First, the "hype" is not externally imposed. It's the genuine reaction of hundreds of millions of people to a technology that would have been considered science fiction just four years ago. The fact that I can't be certain you whether you are an LLM or a human is incredible.

Second, the output of the LLM depends on both its weights and the context. If it has seen something in its context window, it now knows it for all intents and purposes. It learns, in other words.

[xylenox]

Yep, people always forget that early LLMs were sold as "Zero Shot Learning".

[kmacdough]

Sold as learning, but that was a marketing term, not a technical one. From a technical perspective, the LLM is not learning. Only reacting based on its original training.

You might argue that the systems we've built around them are learning in a way, as they strategically condense and save artifacts from past interactions to pass into the LLMs context. But the LLM itself, which is the source of the intelligence, is not learning. It remains entirely unchanged throughout inference. This difference may seem trite, but it has significant impacts over the long term behavior.

[DiogenesKynikos]

You're making a highly arbitrary distinction between learning and ... learning?

The LLM can immediately learn and start using new skills. To any lay person, that's what learning is.