| title | Rethinking RAG in Long Videos: What to Retrieve and How to Use It? |
|---|---|
| authors | Yuho Lee, Jisu Shin, Nicole Hee-Yeon Kim, Jihwan Bang, Juntae Lee, Kyuwoong Hwang, Fatih Porikli, Hwanjun Song |
| date | 2026-06-11 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.13141 |
| affiliation | Data Intelligence System Lab |
| upvotes | 34 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Retrieval-augmented generation is moving beyond text into long, egocentric video, where systems must select query-relevant chunks across multiple modalities and temporal granularities. Yet progress in VideoRAG is limited by two gaps: existing benchmarks allow queries to be answered without the video, obscuring retrieval errors, and prior methods apply a single modality-granularity configuration per query, ignoring chunk-level variability. We address both by introducing V-RAGBench, a benchmark of langlequery, evidence chunk, answerrangle triplets that enables faithful, decoupled evaluation of retrieval and generation, and CARVE, a simple method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk. Each chunk then enters the generator under its winning configuration selected during retrieval, yielding an interleaved evidence form where the chunk-level decision propagates across both stages. CARVE outperforms eight recent VideoRAG baselines, with the chunks supplied to the generator interleaving multiple configurations rather than sharing a single one, a behavior unattainable by query-level methods. |
cite
@misc{lee2026rethinking,
title={Rethinking RAG in Long Videos: What to Retrieve and How to Use It?},
author={Yuho Lee and Jisu Shin and Nicole Hee-Yeon Kim and Jihwan Bang and Juntae Lee and Kyuwoong Hwang and Fatih Porikli and Hwanjun Song},
year={2026},
eprint={2606.13141},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.13141}
}