Hacker News new | ask | show | jobs
by ACCount37 50 days ago
You can logit distill (full token probabilities) or one hot distill (chat logs), or even align hidden states. All are distillation methods.