|
|
|
|
|
by orbital-decay
5 days ago
|
|
Distillation is absolutely not the reason they're good. It's not necessarily even done on a more capable model. It can even be done on itself and still bring improvement, or on a weaker model as well (see GLM and Gemini, which is definitely true because it repeats Deepmind's injections). |
|
If using Anthropic models for distillation doesn't make them better, why do it?