titleR^3-Bench: LLMs Struggle with Resource-Rational Reasoning under Shared Budgets
authorsPeisong Wang, Zhiwei Ma, Bowen Liu, Feixue Liu, Aochuan Chen, Chenyi Zi, Hongchuan Zeng, Yuhan Li, Jia Li
date2026-08-17
venuearXiv (preprint, not peer reviewed)
idarXiv:2608.16033
affiliationData Science & Artificial Intelligence Lab at HKUST
upvotes17 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractIn cognitive science, resource rationality asks how an agent should allocate limited computation to maximize expected value. Most reasoning and agent benchmarks use independent per-task budgets; existing shared-budget studies do not calibrate suite performance against the same model's demonstrated single-problem competence. We introduce R^3-Bench, which evaluates six-problem suites under shared budgets across mathematics, competitive programming, and abstract reasoning in tool-free and agentic settings. Matched single-problem response curves define an offline empirical oracle over observed successes. Across 72 main-table cells for six models, the oracle mean matches or exceeds the contest mean in all cells and is strictly higher in 71. Under moderate tool-free pressure, equal-allocation replay also exceeds contest performance for four of six models. Trajectory diagnostics reveal limited strategy updating and pressure-dependent failure patterns. In a three-model diagnostic under strong agentic pressure, at least one fixed scheduler exceeds the contest mean in six of nine cells, but no policy dominates across domains. These results expose a persistent gap between demonstrated competence and shared-budget realization.

cite

@misc{wang2026r,
  title={R^3-Bench: LLMs Struggle with Resource-Rational Reasoning under Shared Budgets},
  author={Peisong Wang and Zhiwei Ma and Bowen Liu and Feixue Liu and Aochuan Chen and Chenyi Zi and Hongchuan Zeng and Yuhan Li and Jia Li},
  year={2026},
  eprint={2608.16033},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2608.16033}
}