on-policy distillation
on-policy distillation Articles
Browse 3 articles about on-policy distillation.
Can You Distill a 27B LLM Into a 4B Model at Home?
A hands-on test of three LLM distillation methods, training a 4B Qwen model from a 27B teacher on local GPUs, with real benchmark results.
LLM distillationQwen distillationon-policy distillation
Did Moonshot's Kimi K3 Distill Claude or GPT? Examining the Claims
A technical look at whether Moonshot's Kimi K3 could have been distilled from US frontier models, based on how distillation actually works.
Kimi K3 distillationMoonshot AI distillationChinese AI models distillation
What Is Model Distillation in AI? Teacher-Student Training Explained
A clear breakdown of AI model distillation: soft labels, Hinton's original method, on-policy distillation, and why the term gets misused in AI news.
what is distillation AImodel distillation explainedteacher student model