Benchmarks

AIAI-generated summary Unite.AI explains that class imbalance occurs when a target class is much rarer than another, allowing standard training and aggregate metrics to neglect important cases. The guide outlines operational boundaries and metrics to prevent high accuracy from concealing the systematic neglect of minority classes in statistical learning.

AIAI-generated summary HPC-Bench is a performance-grounded benchmark introduced to evaluate large language models as optimizers of computational kernels in realistic high-performance computing workloads. The benchmark includes 114 workloads across 16 high-performance computing computational motifs and provides an automated evaluation pipeline with correctness-gated metrics.

AIAI-generated summary Researchers from Alibaba and several Asian universities created a coding-agent benchmark called Co-Design Bench using real user change requests. GPT-6 Astra achieved the highest score among ten models tested, but fully completed only 26.4 percent of the assigned tasks.

AIAI-generated summary Thinking Machines' Inkling 2 Small model swears in its internal chain of thought during benchmark runs, according to researchers at Vals AI. The researchers found variations of profanity in 50 runs totaling 316 instances, with ProgramBench experiencing the highest impact.

AIAI-generated summary Vals AI reports that autonomous AI agents can infiltrate realistic corporate networks without human assistance. The Vals CyberOffense benchmark shows that open-weight models achieve competitive cyber performance compared to closed systems while costing significantly less.

AIAI-generated summary Science's Last Exam is a new benchmark where an AI agent receives a research goal and tools to pick methods, run experiments, and receive grades from hidden checks. The tasks span nine disciplines built by researchers from Harvard, MIT, Stanford, Oxford, Princeton, and Oak Ridge, with a wet-lab round planned next.

AIAI-generated summary A new benchmark called Into the Unknown tested nine AI agents on open research questions from US national labs. The authors report that none of the 135 submissions met all requirements and expert standards.

AIAI-generated summary Washington and Beijing need to establish a common way to measure the progress of artificial intelligence before deciding how to respond. The next US-China talk on the technology should begin by asking whether both governments understand how rapidly its capabilities are improving.

AIAI-generated summary Patronus.ai measured the fraction of filler words in the reasoning of 12 open models on 20 agentic coding tasks across SWE-bench Verified, SWE-bench Pro and Terminal-Bench 2.1. Patronus.ai says newer releases write fewer filler words, with DeepSeek halving its filler share between April and October builds of its Flash model.

AIAI-generated summary In AA-PersonalityBench, Qwen3.8 27B writes the longest responses of any model measured, averaging about 1,030 words per answer. Average response lengths vary widely across the models measured, ranging from 174 words for GPT-6 Sol to about 1,030 words for Qwen3.8 27B.

AIAI-generated summary OpenAI says it introduced BioBasicsBench, a new benchmark developed with Ligo to measure how well artificial intelligence agents interpret raw experimental data across 100 routine wet-lab tasks. OpenAI says that at the xhigh setting, GPT-6 Astra earned an average of 66.7 percent of the available credit, compared with 43.8 percent for GPT-5.6 Sol.

AIAI-generated summary A new research paper examines how infrastructure arrangements distribute authority and shape whether participants can evaluate artificial intelligence systems independently. The authors introduce the PAAI framework to analyze power, access, adequacy, and independence across twelve African evaluation projects and international joint testing exercises.

AIAI-generated summary Artificial Analysis launched VIALS-AA, a leaderboard for the Handshake AI VIALS benchmark evaluating visual reasoning in life sciences and biotech research. The evaluation uses Stirrup with code execution and image inspection tools to test models across 161 tasks spanning eight domains.

AIAI-generated summary Command Line expanded its ThinkingBox evaluation to 18 models and published results in a paper. The post discusses model evaluations, the price of dependable runs, and reinforcement learning.

AIAI-generated summary Hugging Face is introducing FoundationWorlds as a framework for creating controlled, simulated environments to study robotics. The first foundation world is ToyBlocks, an open simulation framework built around a robot arm and blocks on a tabletop.

AIAI-generated summary Instructing GPT-5 to answer as of a past date reduces the share of answers claiming prior knowledge from 39% to 1%, while its accuracy changes from 76% to 75%. A benchmark called FinDendro posted to arXiv on Nov. 4 by a 15-author team tests safeguards against data leakage using 2,226 dated U.S. facts.

AIAI-generated summary A new benchmark harness tests artificial intelligence models by having them play the social deception game Blood on the Clocktower against each other. The creator spent about twenty thousand dollars of API credits to run matches, revealing performance differences between models like Claude Opus 4.6 and GPT-6 Astra.

AIAI-generated summary Artificial Analysis released Endpoint Accuracy Index v1.1 alongside expanded coverage for GLM-5.3 Flash and Muse Spark Open. The benchmark measures how much accuracy each serverless endpoint preserves by scoring them against a self-hosted reference deployment.

AIAI-generated summary Visa tested 22 artificial intelligence models from 12 labs through realistic shopping scenarios and discovered that 42.8% of the test runs failed. Visa created the Visa Agentic Commerce Benchmark as an industry benchmark designed to provide a roadmap for better agentic shopping experiences.

AIAI-generated summary Artificial Analysis congratulated xdotli and the team at benchflow_ai on the launch of FrontierPhysics. Artificial Analysis stated it is excited to partner to host this evaluation soon.