| title | Self-Distillation with Verbal Regularization |
|---|---|
| authors | Jiazheng Li, Yawei Li, Junru Lu, Zhenyi Shen, Di Yin, Lin Gui, Xing Sun, David Rügamer, Xinpeng Wang, Yulan He |
| date | 2026-10-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.16354 |
| affiliation | King's College London |
| upvotes | 32 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | On-policy self-distillation (OPSD) trains a large language model using supervision from a teacher instantiated from the same model as the student. The teacher is conditioned on privileged information (PI), such as verified solutions. The student, which does not observe PI, learns from the teacher's next-token distributions along its own sampled responses. However, PI can bias supervision toward shortcut reasoning with limited verification or reconsideration, accompanied by a sharp drop in the entropy of the output distribution. To address this challenge, we introduce self-distillation with verbal regularization (VRSD), which uses system prompts to regulate how the teacher uses PI, thereby encouraging rethinking and exploration to counteract premature commitment. Theoretically, we model prompting as attenuating the influence of PI on the teacher distribution and derive conditions for entropy preservation. Across three models and four tasks, VRSD improves Pass@1 over OPSD in 11 of 12 model-task settings, by 3.9 percentage points on average, while generally maintaining higher token entropy. |
cite
@misc{li2026self,
title={Self-Distillation with Verbal Regularization},
author={Jiazheng Li and Yawei Li and Junru Lu and Zhenyi Shen and Di Yin and Lin Gui and Xing Sun and David Rügamer and Xinpeng Wang and Yulan He},
year={2026},
eprint={2610.16354},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.16354}
}