| title | MemoBench: Benchmarking World Modeling in Dynamically Changing Environments |
|---|---|
| authors | Haoyu Chen, Kaichen Zhou, Hang Hua, Kaile Zhang, Jingwen Qian, Wufei Ma, Haonan Chen, Chunjiang Liu, Yizhou Zhao, Xiaoyuan Wang, Weiyue Li, Alan Yuille, Paul Pu Liang, Yilun Du |
| date | 2026-06-25 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.27537 |
| affiliation | Kempner Institute |
| upvotes | 5 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| primer | World models |
| abstract | Video generation models aspire to simulate dynamic environments, and several benchmarks now evaluate memory consistency across frames. However, most assess consistency only while the target remains in view, and the few that force objects out of view evaluate static scenes where nothing changes during occlusion. To bridge this gap, we introduce MemoBench, a diagnostic benchmark built around the disappear-and-reappear paradigm in dynamically changing environments: a target object undergoes a physical process, disappears from view, and must be correctly recovered in its updated state upon reappearance. We curate 360 ground-truth clips spanning synthetic and real-world scenes, and design an evaluation suite combining automated metrics with VQA-based assessment across four diagnostic pillars. Evaluation of eight state-of-the-art models reveals key insights and open challenges regarding memory consistency under the disappear-and-reappear paradigm. |
cite
@misc{chen2026memobench,
title={MemoBench: Benchmarking World Modeling in Dynamically Changing Environments},
author={Haoyu Chen and Kaichen Zhou and Hang Hua and Kaile Zhang and Jingwen Qian and Wufei Ma and Haonan Chen and Chunjiang Liu and Yizhou Zhao and Xiaoyuan Wang and Weiyue Li and Alan Yuille and Paul Pu Liang and Yilun Du},
year={2026},
eprint={2606.27537},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.27537}
}