Agent Evaluation · Sat 7 Nov

AIAI-generated summary A new study by Andy Hall examines how AI models can identify problematic behavior by other AI agents in logs but frequently flag harmless actions as suspicious. The research tested models including GPT-5, GPT-6 Astra, Opus 4.8, and Gemini 3.8 Flash using datasets from METR and Shanghai Jiao Tong University.

Anthropic Red Teaming Hallucinations Agent Evaluation

AIAI-generated summary Science's Last Exam is a new benchmark where an AI agent receives a research goal and tools to pick methods, run experiments, and receive grades from hidden checks. The tasks span nine disciplines built by researchers from Harvard, MIT, Stanford, Oxford, Princeton, and Oak Ridge, with a wet-lab round planned next.

Benchmarks Agent Evaluation AI for Science

AIAI-generated summary A new benchmark called Into the Unknown tested nine AI agents on open research questions from US national labs. The authors report that none of the 135 submissions met all requirements and expert standards.

Code Agents Benchmarks Agent Evaluation AI for Science