| title | Twice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era |
|---|---|
| authors | Ilya Gusev |
| date | 2026-11-19 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.26286 |
| upvotes | 9 (Hugging Face Daily Papers, 2026-11-21) |
| links | abs pdf hf code |
| cite | |
| abstract | Peer review relies on several readers finding different problems. We measure critique overlap in conference and journal reviews. At ICLR, measured overlap rose from 12.0% in 2026 to 21.3% in 2027, in reviews from the same stage of the review process, with year-on-year rises at NeurIPS, ICML and the F1000 and PLOS journals. A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026. We apply MELD, a detector of LLM-generated text, to all reviews. Within a paper, two reviews that MELD flags overlap more than two unflagged reviews: by 3.0 points at ICLR in 2024-2025, by 4.5 points in 2026-2027, and by 5.4 points across five journal sources. These results provide evidence that LLM-flagged reviews of one paper are less distinct from each other than unflagged reviews, and suggest that the diversity of critiques among reviews of a single paper is falling. |
bibtex
@misc{gusev2026twice,
title={Twice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era},
author={Ilya Gusev},
year={2026},
eprint={2611.26286},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.26286}
}