titleContinually Post-training Small Models via Synthetic Self-distillation
authorsWoosung Koh, Hyunsoo Lee, Kyungjae Lee, Haeju Park, Dahyun Lee, Dasol Hwang, Moontae Lee
date2026-10-09
venuearXiv (preprint, not peer reviewed)
idarXiv:2610.13113
affiliationLG AI Research
upvotes39 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf
abstractContinual post-training faces a dual challenge of acquisition and retention: models must learn new capabilities while preserving prior knowledge. While Self-Distillation Fine-Tuning (SDFT) mitigates this using self-supervision with in-context demonstrations, it relies on strong in-context learning and instruction-following abilities. Consequently, SDFT fails on smaller models critical for low-latency, on-device use. To enable continual learning at smaller scales, we first identify the root cause of failure. We show that poor acquisition and forgetting correlate linearly with the post-trained model's distribution shift, revealing the necessity of in-distribution learning. We thus introduce Synthetic Self-Distillation (SynthSelf), imitating SDFT's in-distribution rewrite by outsourcing the task to a minimally larger, same-family aligner. Coupled with off-policy supervision and rejection sampling, this provides the high-quality supervision small models require without breaking their original distribution. Evaluations spanning 4 acquisition datasets (including a mixture) and 14 retention benchmarks show SynthSelf consistently outperforms 6 methods (including off- and on-policy distillation) across 3 small model scales. Extensive further analyses confirm SynthSelf paves the way toward effective small-model continual learning.

cite

@misc{koh2026continually,
  title={Continually Post-training Small Models via Synthetic Self-distillation},
  author={Woosung Koh and Hyunsoo Lee and Kyungjae Lee and Haeju Park and Dahyun Lee and Dasol Hwang and Moontae Lee},
  year={2026},
  eprint={2610.13113},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.13113}
}