|
|
|
|
|
|
by sailingparrot
700 days ago
|
|
|
There is nothing that requires you to train with the same training set, or to re-do RLHF.
You can train on fineweb, and llama 3.1 will learn to use your new tokenizer just fine. There is 0 doubt that you are better of finetuning that model to use your tokenizer than training from scratch. So what Meta gives you for free massively helps you building your model, that's OSS to me. |
|
|