Hacker News new | ask | show | jobs
by okdood64 1125 days ago
New to ML here, what’s the difference between parameters and context?
3 comments

Parameters is like the number of neurons in your brain

Context is how much short term memory you can retain at any one time (think how many cards you can remember the order of in a deck of cards)

Paramters - number of internal variables/weights in the model

Context - Length of input/output buffer (number of input/output tokens possible).

Other answers are already good, just offering yet another difference.

Parameters is something that gets set indirectly via training, it's kept within the weights of the model itself.

Context is what you as a user passes to the model when you're using it, it decides how much text you can actually pass it.

Being able to pass more context means you can (hopefully) make it understand more things that wasn't part of the initial training.