| title | Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents |
|---|---|
| authors | Haoran Zhang, Yifu Lu, Boyang Wang, Xuhui Kang, Yen-Ling Kuo, Zezhou Cheng, Mengdi Wang, Odest Chadwicke Jenkins |
| date | 2026-06-22 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.23085 |
| affiliation | University of Michigan |
| upvotes | 14 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| primer | World models |
| abstract | Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored. Detecting failures in long-horizon robotic tasks is particularly challenging because failure onset is often ambiguous and dense temporal annotations are typically unavailable. We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model. Foresight is trained using only final task-level success or failure labels. By leveraging predictive world-model embeddings, our method provides a unified framework for failure detection across different policies. We further use functional conformal prediction (FCP) to calibrate detection thresholds adaptively. We evaluate Foresight with state-of-the-art vision-language-action policies in simulation on LIBERO-Long, ManiSkill-Long, and BEHAVIOR-1K, compare it against state-of-the-artfailure detection methods, and validate it on real robots with three long-horizon tasks on a ReactorX-200 arm and one task on a Franka arm. Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation. |
cite
@misc{zhang2026foresight,
title={Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents},
author={Haoran Zhang and Yifu Lu and Boyang Wang and Xuhui Kang and Yen-Ling Kuo and Zezhou Cheng and Mengdi Wang and Odest Chadwicke Jenkins},
year={2026},
eprint={2606.23085},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.23085}
}