| title | Retrieval from Within: An Intrinsic Capability of Attention-Based Models |
|---|---|
| authors | Elad Hoffer, Yochai Blau, Edan Kinderman, Ron Banner, Daniel Soudry, Boris Ginsburg |
| date | 2026-05-08 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.05806 |
| affiliation | NVIDIA |
| upvotes | 10 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own internal representations. We introduce INTRA (INTrinsic Retrieval via Attention), a framework where decoder attention queries score pre-encoded evidence chunks that are then directly reused as context for generation. By construction, INTRA unifies retrieval and generation, eliminating the retriever-generator mismatch typical of RAG pipelines. This design also amortizes context encoding by reusing precomputed encoder states across queries. On question-answering benchmarks, INTRA outperforms strong engineered retrieval pipelines on both evidence recall and end-to-end answer quality. Our results demonstrate that attention-based models already possess a retrieval mechanism that can be elicited, rather than added as an external module. |
cite
@misc{hoffer2026retrieval,
title={Retrieval from Within: An Intrinsic Capability of Attention-Based Models},
author={Elad Hoffer and Yochai Blau and Edan Kinderman and Ron Banner and Daniel Soudry and Boris Ginsburg},
year={2026},
eprint={2605.05806},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.05806}
}