| title | Bridged Self-Distillation for On-Policy Learning |
|---|---|
| authors | Runsheng Chen, Yihong Liu, Raoyuan Zhao, Michael A. Hedderich, Hinrich Schütze, Matthias Grabmair |
| date | 2026-10-19 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.26397 |
| affiliation | Technical University of Munich |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | On-policy distillation can transfer the reasoning capabilities of a strong teacher to a smaller student. However, distillation from a stronger teacher model faces two mismatches: its solutions may be difficult for a smaller student to imitate, and its token-level probabilities may be incompatible with, or even unavailable to, the student. Existing methods handle these settings through different forms of supervision. To this end, we propose Bridged Self-Distillation (BSD), a unified framework for distilling from any teacher that provides reasoning traces. The model to be distilled from, termed the external teacher, first provides a reasoning trace as privileged context, which can be generated in an offline fashion. The student generates rollouts without this context, while a copy of the student conditioned on it, serving as the self-teacher, scores the same rollouts in the student's own token space. BSD uses the resulting token-level guidance to adjust updates on unsuccessful rollouts, with a verifier providing the outcome signal. As the external teacher is queried only to produce text, the same training procedure applies to all teacher scenarios: shared-tokenizer, cross-tokenizer, and closed API teachers. Across these scenarios, BSD improves over the base model and outperforms baselines tailored to each scenario, offering a novel and unified pathway for distillation. Code is available: https://github.com/TUMLegalTech/Bridged-Self-Distillation |
cite
@misc{chen2026bridged,
title={Bridged Self-Distillation for On-Policy Learning},
author={Runsheng Chen and Yihong Liu and Raoyuan Zhao and Michael A. Hedderich and Hinrich Schütze and Matthias Grabmair},
year={2026},
eprint={2610.26397},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.26397}
}