|
|
|
|
|
|
by joaogui1
193 days ago
|
|
|
During pre-training the model is learning next-token prediction, which is naturally additive. Even if you added DEL as a token it would still be quite hard to change the data so that it can be used in a mext-token prediction task
Hope that helps |
|
|