|
|
|
|
|
|
by linkregister
19 days ago
|
|
|
Modern pretraining also consists of expensive human-led specialized task creation and grading loops. Synthetic generation and distillation from previous models is another input for training. I wonder how much new text contributes beyond keeping knowledge up-to-date. |
|
|