[worldsayshi]

2048 words?

[wongarsu]

Tokens. Short or common words tend to be one token, while less common words are composed of multiple tokens. For GPT OpenAI gives the rule of thumb that on average you need four tokens to encode three words, and LLaMA should be similar

[worldsayshi]

Well that's for sure bigger than my context size.

[doctoboggan]

2048 "tokens", where one token is roughly equivalent to ¾ of a word

[teaearlgraycold]

Tokens