titleAutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher
authorsGuoxin Chen, Shuang Sun, Fanzhe Meng, Xinyu Tang, Ruihua Song, Wayne Xin Zhao
date2026-11-19
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.26510
affiliationRUC-AIBOX
upvotes10 (Hugging Face Daily Papers, 2026-11-21)
linksabs pdf hf code
cite
relatedRing-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
Evo-Bench: Can Language Models Improve Agent Harness?
SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization
Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards
abstractTask success alone does not specify which behaviors post-training should reinforce. Even successful teacher trajectories can contain unsupported assumptions, unproductive repetition, or superficial verification. We propose Skill-FT (Skill-Guided Fine-Tuning), which uses natural-language skills to suppress undesirable habits and elicit underrepresented effective behaviors during teacher rollouts, then fine-tunes a target model on the resulting trajectories. We introduce AutoMaster, an autonomous post-training system in which an LLM agent develops these interventions through trajectory analysis, skill design, data selection, and feedback from rollouts and target-model evaluation. Across Terminal, Search, and software engineering tasks with 4B and 9B target models, AutoMaster improves mean performance over Vanilla SFT on successful unguided teacher trajectories by 1.42-8.99 percentage points, with the teacher, task pool, training set size, and SFT recipe held fixed. Together with the ablations and trajectory analyses, these results suggest that behavioral shaping can make a fixed model a more effective teacher. We further find that retaining skill instructions during SFT yields higher mean performance than removing them across all six settings, despite their absence at evaluation and identical supervised outputs. These findings suggest that preserving skill context as part of a training demonstration can support better behavioral internalization.

bibtex

@misc{chen2026automaster,
  title={AutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher},
  author={Guoxin Chen and Shuang Sun and Fanzhe Meng and Xinyu Tang and Ruihua Song and Wayne Xin Zhao},
  year={2026},
  eprint={2611.26510},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.26510}
}