| title | RISE: Recursive Improvement via Self-Extrapolating Policy Distillation |
|---|---|
| authors | Yang Li, Semih Yavuz, Shafiq Joty |
| date | 2026-09-04 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.05295 |
| affiliation | Salesforce AI Research |
| upvotes | 18 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | On-policy distillation (OPD) provides dense, per-token supervision for language model post-training, but its effectiveness is bottlenecked by teacher quality: external teachers suffer from distribution mismatch, while self-distillation with privileged conditioning is limited by in-context learning capacity. We propose RISE (Recursive Improvement via Self-Extrapolating Policy Distillation), which constructs a synthetic teacher directly from the model's own RLVR training trajectory. By extrapolating the displacement between the current checkpoint and a trailing anchor---in parameter space or output logit space---RISE converts a sparse outcome-induced parameter update into a dense token-level target, without any external model or privileged conditioning. RISE combines RLVR and OPD in a complementary loop: outcome rewards ground the extrapolation toward correct reasoning, while the extrapolated teacher refines token-level decisions. Moreover, since the teacher is refreshed every iteration as the student improves, distillation becomes a recursive improvement mechanism rather than a one-shot compression step. Experiments spanning mathematical reasoning, multi-domain STEM, code generation, and multi-turn agentic tasks show that RISE outperforms RLVR-only training and on-policy self-distillation across all settings. |
cite
@misc{li2026rise,
title={RISE: Recursive Improvement via Self-Extrapolating Policy Distillation},
author={Yang Li and Semih Yavuz and Shafiq Joty},
year={2026},
eprint={2609.05295},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.05295}
}