| title | What Does an Evaluation License? A Commit-Bound Census of Claim Replay in Inspect Evals |
|---|---|
| authors | Xi Qin, Jizhou Tong |
| date | 2026-08-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.19269 |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Benchmarks can run without determining what their results license. We freeze a large evaluation collection and attempt to replay its historical claims. Most units stop because the evidence required for replay is not bound. Where replay is possible, different claims remain stable at different resolutions. We make this otherwise implicit inference step explicit and executable. |
cite
@misc{qin2026what,
title={What Does an Evaluation License? A Commit-Bound Census of Claim Replay in Inspect Evals},
author={Xi Qin and Jizhou Tong},
year={2026},
eprint={2608.19269},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.19269}
}