Interactive Visual Gym: Evaluating Visual Interaction Models Through Simulation
Salesforce — Learn how Salesforce AI Research built Interactive Visual Gym, an open benchmark using dynamic simulation to evaluate real-time visual AI assistants.
Benchmarks
Benchmarks are standardized tests and scored suites used to measure artificial intelligence capabilities and performance across tasks. Truncated covers new evaluation methods, dataset updates, scoring controversies and how industry labs measure progress.
Humanoid Robot Makers Track Output but Not Payback
PYMNTS.com — The robotics industry has measured progress by machines shipped, tasks demonstrated and hours logged. China’s AGIBOT is proposing a different benchmark:
The Evaluation Framework: The Most Underbuilt Part Of An AI System
Forbes — Together, they answer a practical question: what evidence is enough to say that the next version is better?
In AA-PersonalityBench, MiMo-V2.6-Pro ends 34% of its responses with a question, the highest share of any model measure…
GlobeBench: A Benchmark for Text-Based World Models
patronus.ai — GlobeBench tests whether language models can stand in for an environment by predicting its response to an agent's next action.
A $100 budget buys 40 Intelligence Index tasks (a mix of task archetypes) with Opus 5.5 at medium reasoning effort, or…
Agent Arena measures complex, real-world tasks.
StateBench: Testing how language models track state changes over long documents
huggingface.co — A Blog post by Ai2 on Hugging Face
No AI agent scores above 12 percent on SemiAnalysis' new real-world work benchmark
The Decoder — SemiAnalysis is putting AI agents through simulated knowledge work in its new GDPx benchmark, covering everything from chip design to quant trading. No model breaks 12 percent. Claude Opus 5.5 leads narrowly over Gemini 4 Carbon. Critics point to the small number of tasks per domain and uneven testing conditions.
Why AI's Next Breakthrough Depends On Measurement, Not Just Scale
Forbes — The gap between what benchmarks measure and what businesses need is not theoretical, and it does not live where most leadership teams expect.
AI agents went shopping for TVs and laptops—and messed up nearly 50% of the time, study shows
Fortune — In a first of its kind experiment, Gemini and Muse agents scored the highest when it come to carrying out tasks like buying a TV.
GPT Image 2.6 Sunburst and Flare by @OpenAI are here, running almost 2x faster than GPT Image 2.5.
Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline perf…
New open-source benchmark reveals reasoning as the biggest bottleneck for AI in cloud operations
The Decoder — CloudOpsBench is a new open-source benchmark that measures how well AI agents handle typical DevOps tasks in the cloud, from fixing services and removing resources to restricting access rights, rolling out updates, and decommissioning projects.
In AA-PersonalityBench, the top open weights models on average rank lower than the top proprietary models on truthfulne…
We’ve updated the six Artificial Analysis Capability Indexes to v2.0, aligning them with evaluations from the Artificia…
We’ve worked with the CritPt authors to finalize the CritPt v2 grading server and will be making this available to the…
Rapid LLM development is outrunning the research meant to explain it, study finds
The Decoder — Stanford researchers tested 50 widely cited claims about LLMs against 20 current models. 44 percent of the findings are already "expired," and another 34 percent have weakened significantly. The average lifespan is just 15.7 months. Claims about what models can't do expire fastest.
CWE-bench v2: More coverage, quality, and transparency
huggingface.co — We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Introducing Physical-Engineering’s Last Exam for agents
Google Cloud Blog — Discover PLE, a new benchmark evaluating frontier AI agents on real-world mechanical, electrical, civil, and structural engineering tasks.
Dismiss email signup
Subscribe to 
The top AI news in your email delivered each morning. Choose categories