| title | WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark |
|---|---|
| authors | Yida Yin, Harish Krishnakumar, Chung Peng Lee, Boya Zeng, Wenhao Chai, Shengbang Tong, Wenhu Chen, Hu Xu, Xingyu Fu, Gabriel Sarch, Aleksandra Korolova, Zhuang Liu |
| date | 2026-06-04 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.06538 |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | In real-world applications, models are expected to perform reliably across diverse settings. Yet, many existing multimodal benchmarks expand task types without capturing the visual diversity needed to handle open-ended visual inputs. We present WorldBench, a challenging and visually diverse reasoning benchmark to evaluate Multimodal Large Language Models (MLLMs). We build a taxonomy of thousands of visual concepts across multiple domains (e.g., living things). Guided by this taxonomy, we curate a broad collection of images from search engines and existing datasets to comprehensively represent the visual world. Through structured trial-and-error, we manually design challenging questions that frontier MLLMs fail to answer. On quantitative and human evaluations, WorldBench achieves higher visual diversity than any existing diverse benchmark. Evaluating 15 MLLMs on WorldBench reveals weaknesses in visual understanding: even the strongest model reaches only 64.0% accuracy, while some models perform marginally above chance-level. We hope our work highlights the importance of visual diversity in building multimodal benchmarks. |
cite
@misc{yin2026worldbench,
title={WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark},
author={Yida Yin and Harish Krishnakumar and Chung Peng Lee and Boya Zeng and Wenhao Chai and Shengbang Tong and Wenhu Chen and Hu Xu and Xingyu Fu and Gabriel Sarch and Aleksandra Korolova and Zhuang Liu},
year={2026},
eprint={2606.06538},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.06538}
}