Benchmarks & Evaluation

AIAI-generated summary Researchers from Alibaba and several Asian universities created a coding-agent benchmark called Co-Design Bench using real user change requests. GPT-6 Astra achieved the highest score among ten models tested, but fully completed only 26.4 percent of the assigned tasks.

AIAI-generated summary Science's Last Exam is a new benchmark where an AI agent receives a research goal and tools to pick methods, run experiments, and receive grades from hidden checks. The tasks span nine disciplines built by researchers from Harvard, MIT, Stanford, Oxford, Princeton, and Oak Ridge, with a wet-lab round planned next.

AIAI-generated summary A new benchmark called Into the Unknown tested nine AI agents on open research questions from US national labs. The authors report that none of the 135 submissions met all requirements and expert standards.

AIAI-generated summary Washington and Beijing need to establish a common way to measure the progress of artificial intelligence before deciding how to respond. The next US-China talk on the technology should begin by asking whether both governments understand how rapidly its capabilities are improving.

AIAI-generated summary Patronus.ai measured the fraction of filler words in the reasoning of 12 open models on 20 agentic coding tasks across SWE-bench Verified, SWE-bench Pro and Terminal-Bench 2.1. Patronus.ai says newer releases write fewer filler words, with DeepSeek halving its filler share between April and October builds of its Flash model.

AIAI-generated summary Qwen-Audio-3.1-TTS-Plus leads in US English according to the analysis. HeyGen Voice performs better in UK English and also leads the overall leaderboard.

AIAI-generated summary In AA-PersonalityBench, Qwen3.8 27B writes the longest responses of any model measured, averaging about 1,030 words per answer. Average response lengths vary widely across the models measured, ranging from 174 words for GPT-6 Sol to about 1,030 words for Qwen3.8 27B.

AIAI-generated summary OpenAI says it introduced BioBasicsBench, a new benchmark developed with Ligo to measure how well artificial intelligence agents interpret raw experimental data across 100 routine wet-lab tasks. OpenAI says that at the xhigh setting, GPT-6 Astra earned an average of 66.7 percent of the available credit, compared with 43.8 percent for GPT-5.6 Sol.

AIAI-generated summary A new research paper examines how infrastructure arrangements distribute authority and shape whether participants can evaluate artificial intelligence systems independently. The authors introduce the PAAI framework to analyze power, access, adequacy, and independence across twelve African evaluation projects and international joint testing exercises.

AIAI-generated summary Artificial Analysis launched VIALS-AA, a leaderboard for the Handshake AI VIALS benchmark evaluating visual reasoning in life sciences and biotech research. The evaluation uses Stirrup with code execution and image inspection tools to test models across 161 tasks spanning eight domains.

AIAI-generated summary Command Line expanded its ThinkingBox evaluation to 18 models and published results in a paper. The post discusses model evaluations, the price of dependable runs, and reinforcement learning.

AIAI-generated summary Instructing GPT-5 to answer as of a past date reduces the share of answers claiming prior knowledge from 39% to 1%, while its accuracy changes from 76% to 75%. A benchmark called FinDendro posted to arXiv on Nov. 4 by a 15-author team tests safeguards against data leakage using 2,226 dated U.S. facts.

AIAI-generated summary Egyptian and Nigerian English have joined the Open ASR Leaderboard through a new collaboration with Liva AI. Liva AI says it collects multimodal interaction data through consumer products used by real people.

AIAI-generated summary A new benchmark harness tests artificial intelligence models by having them play the social deception game Blood on the Clocktower against each other. The creator spent about twenty thousand dollars of API credits to run matches, revealing performance differences between models like Claude Opus 4.6 and GPT-6 Astra.

AIAI-generated summary Artificial Analysis released Endpoint Accuracy Index v1.1 alongside expanded coverage for GLM-5.3 Flash and Muse Spark Open. The benchmark measures how much accuracy each serverless endpoint preserves by scoring them against a self-hosted reference deployment.

AIAI-generated summary Inkling 2 Small by thinkymachines has entered Code Arena: WebDev with a score of 1654 points evaluated by AutoEval. The model is priced at one dollar per one million output tokens and thirty cents per one million input tokens.

AIAI-generated summary Visa tested 22 artificial intelligence models from 12 labs through realistic shopping scenarios and discovered that 42.8% of the test runs failed. Visa created the Visa Agentic Commerce Benchmark as an industry benchmark designed to provide a roadmap for better agentic shopping experiences.

AIAI-generated summary Artificial Analysis congratulated xdotli and the team at benchflow_ai on the launch of FrontierPhysics. Artificial Analysis stated it is excited to partner to host this evaluation soon.

AIAI-generated summary Salesforce AI Research developed Interactive Visual Gym to evaluate and train visual artificial intelligence assistants in dynamic, closed-loop simulation environments. Salesforce says Gemini 4 Argon achieves a 53.4 percent task success rate averaged across three environments in the IVG-Bench evaluation.

AIAI-generated summary Simform CTO and co-founder Hiren writes that evaluation frameworks are often missing from AI systems, leaving teams unable to explain quality declines or verify fixes. A proper framework requires representative examples, consistent scoring methods and clear release thresholds to ensure changes actually improve outcomes.