| title | Xetrieval: Mechanistically Explaining Dense Retrieval |
|---|---|
| authors | Zhixin Cai, Jun Bai, Yang Liu, Jiaqi Li, Yichi Zhang, Taichuan Li, Zhuofan Chen, Zixia Jia, Zilong Zheng, Wenge Rong |
| date | 2026-05-28 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.29507 |
| affiliation | Beihang University |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Explaining why dense retrievers assign high relevance scores remains challenging because retrieval decisions are made through opaque high-dimensional embeddings. Existing explanations often focus on surface signals, such as lexical matches, token alignments, or post-hoc textual rationales, and thus provide limited insight into the latent factors that shape dense retrieval behavior at the embedding level. We propose Xetrieval, an embedding-level mechanistic framework for explaining dense retrieval. Xetrieval first introduces a lightweight reasoning internalizer that approximates Chain-of-Thought reasoning directly in the embedding space with a single forward pass, enriching sentence embeddings with reasoning-oriented information while avoiding expensive autoregressive generation. It then decomposes these reasoning-enhanced embeddings into sparse, human-interpretable features, each associated with a coherent natural language description. By aggregating sparse feature overlaps across multiple document-side views, Xetrieval provides feature-level explanations of individual retrieval decisions. Experiments on diverse retrievers and benchmarks show that Xetrieval uncovers coherent interpretable features, yields stronger pair-level intervention effects, and supports task-level feature steering. The project page and source code are available at https://hihiczx.github.io/Xetrieval . |
cite
@misc{cai2026xetrieval,
title={Xetrieval: Mechanistically Explaining Dense Retrieval},
author={Zhixin Cai and Jun Bai and Yang Liu and Jiaqi Li and Yichi Zhang and Taichuan Li and Zhuofan Chen and Zixia Jia and Zilong Zheng and Wenge Rong},
year={2026},
eprint={2605.29507},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.29507}
}