New benchmark audits AI peer reviewers with injected errors in real papers
AI-generated summary Researchers published an arXiv preprint describing a benchmark that evaluates AI-based peer reviewers by inserting 357 objectively defined errors into 30 recent arXiv preprints. The study reports that large language models vary in detection performance by up to 50 percentage points depending on model choice and prompt design.