titleImagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
authorsTianxiang Jiang, Linquan Wu, Sheng Xia, Songze Li, Ziang Yan, Haoyu Yang, Yu Qiao, Yi Wang
date2026-06-04
venuearXiv (preprint, not peer reviewed)
idarXiv:2606.05769
affiliationOpenGVLab
upvotes7 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractVideo event prediction (VEP) requires models to infer unobserved future states from partial video evidence. Existing video MLLMs usually verbalize intermediate future reasoning in text space: once visual evidence is verbalized, fine-grained motion, geometry, and interaction cues can be lost, leading to plausible but visually ungrounded hallucinations. We introduce Future-L1, an interleaved latent visual reasoning framework that lets an MLLM alternate between language tokens and continuous latent visual spans during autoregressive decoding. To train this capability, we construct Future-L1-50K by selecting examples where future visual hints help prediction and align latent states to future-frame embeddings, then further optimize sampled latent trajectories with LA-DAPO, a latent-aware RL objective with outcome-contrastive and temporal-diversity rewards. Future-L1 achieves new state-of-the-art results on both benchmarks: on FutureBench, it improves Qwen3-VL-8B from 61.0 to 85.4 and exceeds the previous best Video-CoE by 10.4 points; on TwiFF-Bench, it improves the average score from 2.44 to 3.04. These results suggest that future-oriented video reasoning benefits from preserving intermediate visual semantics in latent space rather than translating every reasoning step into text.

cite

@misc{jiang2026imagine,
  title={Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction},
  author={Tianxiang Jiang and Linquan Wu and Sheng Xia and Songze Li and Ziang Yan and Haoyu Yang and Yu Qiao and Yi Wang},
  year={2026},
  eprint={2606.05769},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2606.05769}
}