|
|
|
|
|
|
by cadamsdotcom
513 days ago
|
|
|
Synthetic data is doing wonders for models like Phi-4, and at least part of the dataset for DeepSeek-R1 came from their earlier models. If you read the literature from the Phi-4 team it talks about synthetic data allowing better control over the training process. The upfront investment is greater but pays off over multiple generations of trained models - and doesn’t leave you with SolidGoldMagikarp ;) https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldm... |
|
|