Hacker News new | ask | show | jobs
by eurekin 1173 days ago
I did try, but got:

``` ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported. ```

2 comments

> Unfortunately there's a mismatch between the model generated by the delta patcher and the tokenizer (32001 vs 32000 tokens). There's a tool to fix this at llama-tools (https://github.com/Ronsor/llama-tools). Add 1 token like (C controltoken), and then run the conversion script.
Just rename it in the tokenconfig.json
Thanks, that indeed worked!

This and using conda in wsl2, instead on bare windows