Benchmarks

AIAI-generated summary Salesforce AI Research developed Interactive Visual Gym to evaluate and train visual artificial intelligence assistants in dynamic, closed-loop simulation environments. Salesforce says Gemini 4 Argon achieves a 53.4 percent task success rate averaged across three environments in the IVG-Bench evaluation.

AIAI-generated summary AGIBOT introduced Worken on Oct. 29 as a new metric to measure effective intelligent work hours for humanoid robots. Agility Robotics and Figure AI publish operating data such as totes moved and factory shifts worked, but the figures cannot be compared with one another or turned into a return.

AIAI-generated summary Simform CTO and co-founder Hiren writes that evaluation frameworks are often missing from AI systems, leaving teams unable to explain quality declines or verify fixes. A proper framework requires representative examples, consistent scoring methods and clear release thresholds to ensure changes actually improve outcomes.

AIAI-generated summary In AA-PersonalityBench, MiMo-V2.6-Pro concludes 34% of its responses with a question, which is the highest share recorded among all evaluated models. The measured share of responses ending with questions spans from 34% down to 0% across the various models, with several models registering 0% or nearly 0%.

AIAI-generated summary Patronus AI introduces GlobeBench, a new benchmark featuring 1,632 tasks designed to evaluate text-based world models across five domains and 19 simulation skills. GPT-6 Astra achieved the highest success rate at 49.57%, while Patronus AI notes that no model solved half of the benchmark tasks.

AIAI-generated summary A one hundred dollar budget purchases forty Intelligence Index tasks using Opus 5.5 at medium reasoning effort or ten tasks at maximum effort. The same amount covers one hundred sixty six tasks with DeepSeek V4.1 Flash 1017 at maximum effort or two thousand five hundred tasks with GPT-6 Luna at medium effort.

AIAI-generated summary Agent Arena measures complex, real-world tasks that span hundreds of turns and amplify differences in frontier models. Comparing top models from five leading labs reveals a fifty-five minute gap in duration, with Claude Opus 5.5 taking the longest and Kimi K3.1 Pro spending the least time.

AIAI-generated summary Hugging introduced StateBench, a benchmark designed to evaluate how language models track entity state changes across long documents. The publisher says tests showed recurrent and hybrid models generalize much farther than transformers when trained directly on these tasks.

AIAI-generated summary Analytics firm SemiAnalysis has released GDPx, a benchmark that places artificial intelligence agents into simulated knowledge work environments to perform professional tasks. Claude Opus 5.5 tops the leaderboard at twelve percent while other models score lower on the new 206 tasks spanning eleven domains.

AIAI-generated summary Vectara released an updated leaderboard from late 2025 revealing that several leading reasoning models exhibited hallucination rates above 10% when summarizing source documents. EY reported that 99% of organizations experienced financial losses from AI-related risks with an average cost of $4.4 million per company.

AIAI-generated summary Visa rolled out a new website making the results of its agent shopping experiment available to the public. The study reveals that personal AI agents like Muse or Claude messed up nearly 50 percent of the time when purchasing TVs and laptops.

AIAI-generated summary @OpenAI has launched GPT Image 2.6 Sunburst and Flare with enhanced speed running almost two times faster than GPT Image 2.5. @OpenAI reports that Sunburst Medium and Flare Medium deliver enhanced photorealism and greater editing consistency while maintaining the lead over competing labs.

AIAI-generated summary Artificial Analysis has launched the next version of its API Performance Benchmarking with Time per Task as the headline performance metric for serverless endpoints. Artificial Analysis says the update reflects how agentic workloads make the time required to complete a task more important than streaming speed.

AIAI-generated summary The team led by Ang Chen at the University of Michigan has released CloudOpsBench version 1.0 as a new open-source benchmark measuring AI agent performance on DevOps tasks. The benchmark reveals that reasoning is the primary bottleneck for AI in cloud operations because models frequently misjudge task scope or stop too early.

AIAI-generated summary Artificial Analysis has launched AA-PersonalityBench to compare the personality traits and metrics of twenty different leading models. Artificial Analysis states that the top open weights models rank lower on truthfulness and higher on slop compared to the top proprietary models.

AIAI-generated summary Artificial Analysis has released version 2.0 of its six Artificial Analysis Capability Indexes with updated evaluations and changed category weights. Artificial Analysis states that Claude Opus 5.5 (max) leads in all six indexes followed by GPT-6 Astra (max) in several areas.

AIAI-generated summary @ArtificialAnlys announced that it will make the CritPt v2 grading server available to the community via its API tomorrow. @ArtificialAnlys stated that it regraded every model and observed small score increases for most models while frontier model rankings remain unchanged.

AIAI-generated summary Stanford researchers Owen Queen and James Zou examined claims using models released between July 2024 and September 2026 by eight different developers. The study found that 44 percent of the tested claims about large language models have already expired.

AIAI-generated summary Collinear AI releases CWE-bench v2, a public set of 100 tasks designed to test the limits of frontier cyber capabilities. Collinear AI says the tasks cover 91 different CWEs and span a wide range of vulnerability classes.

AIAI-generated summary Google AI and Infrastructure releases Physical-Engineering's Last Exam, a benchmark and open competition built in collaboration with Mercor to test frontier models on engineering tasks. Google says the benchmark evaluates agent capabilities across mechanical, electrical, civil, and structural engineering disciplines.