| title | Rubric-based On-policy Distillation |
|---|---|
| authors | Junfeng Fang, Zhepei Hong, Mao Zheng, Mingyang Song, Gengsheng Li, Houcheng Jiang, Dan Zhang, Haiyun Guo, Xiang Wang, Tat-Seng Chua |
| date | 2026-05-08 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.07396 |
| upvotes | 39 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | On-policy distillation (OPD) is a powerful paradigm for model alignment, yet its reliance on teacher logits restricts its application to white-box scenarios. We contend that structured semantic rubrics can serve as a scalable alternative to teacher logits, enabling OPD using only teacher-generated responses. To prove it, we introduce ROPD, a simple yet foundational framework for rubric-based OPD. Specifically, ROPD induces prompt-specific rubrics from teacher-student contrasts, and then utilizes these rubrics to score the student rollouts for on-policy optimization. Empirically, ROPD outperforms the advanced logit-based OPD methods across most scenarios, and achieving up to a 10x gain in sample efficiency. These results position rubric-based OPD as a flexible, black-box-compatible alternative to the prevailing logit-based OPD, offering a simple yet strong baseline for scalable distillation across proprietary and open-source LLMs. Code is available at https://github.com/Peregrine123/ROPD_official. |
cite
@misc{fang2026rubric,
title={Rubric-based On-policy Distillation},
author={Junfeng Fang and Zhepei Hong and Mao Zheng and Mingyang Song and Gengsheng Li and Houcheng Jiang and Dan Zhang and Haiyun Guo and Xiang Wang and Tat-Seng Chua},
year={2026},
eprint={2605.07396},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.07396}
}