| title | Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning |
|---|---|
| authors | Haozhi Yuan, Handong Li, Yifan Zhang, Qianshan Wei |
| date | 2026-11-09 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.12886 |
| affiliation | Nanjing University |
| upvotes | 39 (Hugging Face Daily Papers, 2026-11-12) |
| links | abs pdf hf code |
| cite | |
| related | OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native Tools EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory Omni-Woodpecker: Hallucination Correction via Evidence-Prior Rebalancing for Omni LLMs |
| abstract | Agentic reinforcement learning (RL) and multi-teacher on-policy distillation (MOPD) offer complementary routes to post-training. For visual agents, however, tool use presents two related challenges. Models may overlook useful tool-returned observations, while poorly grounded actions can limit the evidence available for subsequent distillation. We introduce Reframe, a post-training framework that treats the observation state, rather than only the output distribution, as an object of alignment. First, Ground-Drop-GRPO (GD-GRPO) trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of the original visual input, encouraging them to acquire grounded evidence and use the returned observations. Second, Observation-Gated-MOPD (OG-MOPD) replaces deficient tool actions with canonical ones mid-rollout and resumes distillation from the repaired observation. The two stages complement each other, as learning to rely on tool-returned observations can help the policy benefit from observation repair. Neither stage requires a task-specific SFT cold start. Extensive experiments over 14 image and video benchmarks demonstrate that Reframe outperforms both Mix-RL and vanilla MOPD. Ablation results show that GD-GRPO increases observation dependence and tool gain compared with outcome-based GRPO. |
bibtex
@misc{yuan2026reframe,
title={Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning},
author={Haozhi Yuan and Handong Li and Yifan Zhang and Qianshan Wei},
year={2026},
eprint={2611.12886},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.12886}
}