| title | DeepStress: Stress-Testing Deep Search Agents |
|---|---|
| authors | Ismael Rousseau, Geraldine Damnati, Frederic Bechet |
| date | 2026-11-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2607.13920 |
| affiliation | Orange |
| upvotes | 5 (Hugging Face Daily Papers, 2026-11-15) |
| links | abs pdf hf |
| cite | |
| abstract | While search agents demonstrate impressive capabilities in multi-step question answering, their robustness to poor-quality evidence remains under-explored. Such evidence is rare in benchmark corpora, and resulting failures are hard to reproduce. We propose DeepStress, a dynamic stress-testing framework that subjects agents to degraded evidence at controlled rates. It replaces the retrieval module with a synthetic environment where documents are generated dynamically for each query, under independent control of their trustworthiness, relevance, and factuality. To characterize behavior under stress, we go beyond binary accuracy with a five-way classification of final-answer outcomes and two new reliability-aware metrics, RAS and CoPRAS, which credit cautious behavior and account for computational cost. Evaluating 12 systems on questions from HotpotQA and BrowseComp-Plus, we find that several open-weight agents answer confidently when sources contain obvious credibility red flags, and that many HotpotQA questions are answered from parametric knowledge even when all returned documents are factually wrong. |
bibtex
@misc{rousseau2026deepstress,
title={DeepStress: Stress-Testing Deep Search Agents},
author={Ismael Rousseau and Geraldine Damnati and Frederic Bechet},
year={2026},
eprint={2607.13920},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2607.13920}
}