Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
 
research article

The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates

Russo, Giuseppe  
•
Horta Ribeiro, Manoel
•
Davidson, Tim Ruben  
Show more
October 16, 2025
Proceedings of the ACM on Human-Computer Interaction

Journals and conferences worry that peer reviews assisted by artificial intelligence (AI), in particular, large language models (LLMs), may negatively influence the validity and fairness of the peer-review system, a cornerstone of modern science. In this work, we address this concern with a study of the prevalence and impact of AI-assisted peer reviews in the context of the 2024 International Conference on Learning Representations (ICLR), a large and prestigious machine-learning conference. Our contributions are threefold. Firstly, we obtain a lower bound for the prevalence of AI-assisted reviews at ICLR 2024 using the closed- and open-source LLM detectors, estimating that at least 15.8% of reviews were written with AI assistance. Secondly, we estimate the impact of AI-assisted reviews on submission scores. Considering pairs of reviews with different scores assigned to the same paper, we find that in 53.4% of pairs, the AI-assisted review scores higher than the human review (p = 0.002; relative difference in probability of scoring higher: +14.4% in favor of AI-assisted reviews). Thirdly, we assess the impact of receiving an AI-assisted peer review on submission acceptance. In a matched study, submissions near the acceptance threshold that received an AI-assisted peer review were 4.9 percentage points (p = 0.024) more likely to be accepted than submissions that did not. Overall, we show that AI-assisted reviews are consequential to the peer-review process and offer a discussion on future implications of current trends.

  • Details
  • Metrics
Type
research article
DOI
10.1145/3757667
Author(s)
Russo, Giuseppe  

École Polytechnique Fédérale de Lausanne

Horta Ribeiro, Manoel
Davidson, Tim Ruben  

École Polytechnique Fédérale de Lausanne

Veselovsky, Veniamin
West, Robert  

École Polytechnique Fédérale de Lausanne

Date Issued

2025-10-16

Publisher

Association for Computing Machinery (ACM)

Published in
Proceedings of the ACM on Human-Computer Interaction
Volume

9

Issue

7

Start page

1

End page

28

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
DLAB  
FunderFunding(s)Grant NumberGrant URL

Swiss National Science Foundation

TMSGI2_211379

Available on Infoscience
October 20, 2025
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/255066
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés