| title | You don't need long context |
|---|---|
| authors | Peter Belcak, Saurav Muralidharan, Jindong Jiang, Yonggan Fu, Wonmin Byeon, Hanrong Ye, Ryan Angilly, Ali Taghibakhshi, Shih-Yang Liu, Siru Ouyang, Greg Heinrich, Pavlo Molchanov |
| date | 2026-11-03 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.05306 |
| affiliation | NVIDIA |
| upvotes | 9 (Hugging Face Daily Papers, 2026-11-05) |
| links | abs pdf hf |
| cite | |
| abstract | Later tokens are costlier to generate than earlier tokens. On frontier open weight models, the disparity in cost may rise to as much as 7x. Through frequent compaction, we show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts. We further demonstrate that LLM performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context, and based on that we outline the concept of semantic state space models (SSSMs). Our findings question the need for long-context language models. |
bibtex
@misc{belcak2026you,
title={You don't need long context},
author={Peter Belcak and Saurav Muralidharan and Jindong Jiang and Yonggan Fu and Wonmin Byeon and Hanrong Ye and Ryan Angilly and Ali Taghibakhshi and Shih-Yang Liu and Siru Ouyang and Greg Heinrich and Pavlo Molchanov},
year={2026},
eprint={2611.05306},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.05306}
}