titleTwice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era
authorsIlya Gusev
date2026-11-19
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.26286
upvotes9 (Hugging Face Daily Papers, 2026-11-21)
linksabs pdf hf code
cite
abstractPeer review relies on several readers finding different problems. We measure critique overlap in conference and journal reviews. At ICLR, measured overlap rose from 12.0% in 2026 to 21.3% in 2027, in reviews from the same stage of the review process, with year-on-year rises at NeurIPS, ICML and the F1000 and PLOS journals. A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026. We apply MELD, a detector of LLM-generated text, to all reviews. Within a paper, two reviews that MELD flags overlap more than two unflagged reviews: by 3.0 points at ICLR in 2024-2025, by 4.5 points in 2026-2027, and by 5.4 points across five journal sources. These results provide evidence that LLM-flagged reviews of one paper are less distinct from each other than unflagged reviews, and suggest that the diversity of critiques among reviews of a single paper is falling.

bibtex

@misc{gusev2026twice,
  title={Twice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era},
  author={Ilya Gusev},
  year={2026},
  eprint={2611.26286},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.26286}
}