back

AAAI-26 AI Peer Review Pilot: AI Generated Reviews for All 22,977 Submissions, Authors Preferred Them on Technical Accuracy

2026-04-27 13:11

A paper published April 15 documents the first full-scale live deployment of AI peer review at a major conference: every one of 22,977 main-track AAAI-26 submissions received a clearly labeled AI-generated review, all produced in under a day using GPT-5 at high reasoning effort with multi-stage tool use and safeguards. A post-conference survey found authors and program committee members not only found the AI reviews useful but actually preferred them over human reviews on technical accuracy and research suggestions; the system also substantially outperformed a simple LLM baseline on a benchmark for detecting scientific weaknesses. The AI reviews supplemented human reviewers and carried no accept/reject scores—the full review process overview notes that all LLM-generated content was reviewed by human experts before being shared.

Citations