| title | Confidence-Aware Tool Orchestration for Robust Video Understanding |
|---|---|
| authors | Yangfan He, Yujin Choi, Jaehong Yoon |
| date | 2026-06-25 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.26904 |
| affiliation | Nanyang Technological University Singapore |
| upvotes | 12 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Video reasoning language models implicitly assume that every input frame is equally reliable. This leads to what we term the Blind Trust Problem: under realistic perturbations such as motion blur, glare, or occlusion, frontier video reasoning models can suffer 15-30%p accuracy drops on real-world embodied benchmarks, while remaining unaware that their visual evidence has been degraded. To address this challenge, we propose Robust-TO, an agentic video understanding framework that explicitly integrates per-frame trustworthiness into every stage of reasoning. Robust-TO organizes heterogeneous visual perception tools under a unified evidence interface. Each tool receives a sub-query derived from the original question and a set of trustworthy frames selected by the reliability-relevance score. It returns evidence in a shared format: a concrete prediction (e.g., a bounding box, motion trajectory, recognized text, or action label), temporal grounding, and a calibrated reliability score. During reasoning, these calibrated scores guide evidence weighting in a three-tier synthesis process (high/medium/low) and define a confidence-cost GRPO reward that jointly optimizes correctness, evidence reliability, and efficiency. On two video reasoning benchmarks spanning eight tasks, Robust-TO achieves 56.4% average accuracy on clean inputs, surpassing the strongest open-source baseline by 10.6%p and outperforming Gemini-2.5-Pro (46.2%). Under five realistic corruption types, Robust-TO maintains 54.3% average accuracy, 5.8%p above the strongest open-source baseline, while exhibiting the smallest clean-to-corrupted accuracy drop among all compared methods. |
cite
@misc{he2026confidence,
title={Confidence-Aware Tool Orchestration for Robust Video Understanding},
author={Yangfan He and Yujin Choi and Jaehong Yoon},
year={2026},
eprint={2606.26904},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.26904}
}