| title | MULTI3IR: A Benchmark for Multi-perspective Multi-domain Multi-modal Information Retrieval |
|---|---|
| authors | Seokwon Song, Sohyeon Kim, Gunhee Kim |
| date | 2026-08-31 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.30949 |
| affiliation | Seoul National University |
| upvotes | 10 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Information retrieval (IR) increasingly targets open-ended queries that admit diverse perspectives. Existing IR benchmarks, however, focus primarily on closed-ended queries, while even open-ended benchmarks largely consist of queries whose supporting documents span a single subject domain and modality. We introduce Multi^3IR, a benchmark that evaluates how well retrievers cover the multifaceted perspectives of open-ended queries across diverse domains and modalities. It comprises 104.9K Stack Exchange queries, each annotated with perspective descriptions that capture the query's implicit viewpoints. We further propose SPIN, a parameter- and label-efficient method that learns noise vectors to steer embeddings toward diverse yet meaningful semantic directions. Experiments show that existing multimodal retrievers suffer from single-perspective bias, while SPIN substantially improves perspective coverage on Multi^3IR and generalizes well to unseen open-ended IR benchmarks. The dataset and experimental code are available at https://github.com/seokwon99/Multi3IR. |
cite
@misc{song2026multi,
title={MULTI3IR: A Benchmark for Multi-perspective Multi-domain Multi-modal Information Retrieval},
author={Seokwon Song and Sohyeon Kim and Gunhee Kim},
year={2026},
eprint={2608.30949},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.30949}
}