| title | Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training |
|---|---|
| authors | Artyom Sorokin, Nazar Buzun, Alexander Anokhin, Oleg Inozemcev, Egor Vedernikov, Petr Anokhin, Mikhail Burtsev, Trushkov Alexey, Yin Wenshuai, Evgeny Burnaev |
| date | 2026-05-04 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2511.07328 |
| upvotes | 15 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Retrieval-Augmented Generation (RAG) methods enhance LLM performance by efficiently filtering relevant context for LLMs, reducing hallucinations and inference cost. However, most existing RAG methods focus on single-step retrieval, which is often insufficient for answering complex questions that require multi-step search. Recently, multi-step retrieval approaches have emerged, typically involving the fine-tuning of small LLMs to perform multi-step retrieval. This type of fine-tuning is highly resource-intensive and does not enable the use of larger LLMs. In this work, we propose Q-RAG, a novel approach that fine-tunes the Embedder model for multi-step retrieval using reinforcement learning (RL). Q-RAG offers a competitive, resource-efficient alternative to existing multi-step retrieval methods for open-domain question answering and achieves state-of-the-art results on the popular long-context benchmarks BabiLong and RULER for contexts up to 10M tokens. Code is available at https://github.com/griver/Q-RAG |
cite
@misc{sorokin2026q,
title={Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training},
author={Artyom Sorokin and Nazar Buzun and Alexander Anokhin and Oleg Inozemcev and Egor Vedernikov and Petr Anokhin and Mikhail Burtsev and Trushkov Alexey and Yin Wenshuai and Evgeny Burnaev},
year={2026},
eprint={2511.07328},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2511.07328}
}