Y
Hacker News
new
|
ask
|
show
|
jobs
by
minimaxir
220 days ago
There's more than enough ALL CAPS text in the corpus of the entire internet, and enough semantic context associated with it for it to be intended to be in the imperative voice.
1 comments
miohtama
220 days ago
Shouldn't all caps normalised to tokens like low caps? There are no separate tokens for all caps and low caps in Llama, or at least not in the past.
link
minimaxir
220 days ago
Looking at the tokenizer for the older Llama 2 model, the tokenizer has capital letters in it:
https://huggingface.co/meta-llama/Llama-2-7b-hf
link