Media Rating Council sets Q1 2027 draft for first AI measurement standards
ppc.land — Measurement services are already audited on AI under 7 existing MRC rulebooks and 9 principles. Phase two lands by mid-2027, so which AI uses remain uncovered?
Benchmarks & Evaluation
Benchmarks and evaluation encompass the standard tests, datasets and metrics used to measure and compare artificial intelligence system capabilities and performance. Truncated covers research papers, testing methodologies, score debates and industry standards for evaluating models.
S. Korea ranks 10th place in global AI, digital competitiveness
en.yna.co.kr — SEOUL, Nov. 11 (Yonhap) -- South Korea came in 10th place among 70 countries in a global d...
Vals AI Legal Aid Benchmark Finds Models Fall Short on Local Help
www.unite.ai — NextLadder Ventures and Vals AI announced the Access to Justice Benchmark on November 10, 2026, reporting that none of 23 AI models reached its 80% threshold for a reliable first point of contact across 252 legal-aid sce...
Our recent Agent Arena analysis found that Gemini 4 Carbon by @GoogleDeepMind becomes more cost-competitive as tasks ge…
MiniMax H3-Image by @MiniMax_AI is now in the Arena!
Great to partner with @GoogleDeepMind in bringing MKA Benchmark to Artificial Analysis
Best OCR Model for Scanned Documents: 17 Routes Measured
openrouter.ai — We ran 8 scanned pages through 15 vision models, Mistral OCR and Tesseract. The most accurate, the best value, and the made-up values to watch for.
Keenable Search 1.0 takes 1st (pro, 83) and 2nd (realtime, 82) on the Artificial Analysis Search Index: ahead of Exa (a…
Humans outperform AI sixfold in teamwork, a new study from Skillprint finds
gamesbeat.com — Skillprint has released new findings from Signal, its human-AI evaluation platform, revealing AI systems struggle and still require humans.
TimeOmni-Live tests the limits of memory in continuous sensor streams
forkast.news — The model scores strongly on an in-distribution suite, but trails Opus 5 on a separate Real Test split and posts very low scores on some long-horizon tasks.
LLM Reliability Layers Under Test: Accuracy, Yield, and the Limits of Verification
www.infoq.com — Verification layers sort outputs by trust rather than fixing errors. In a benchmark of six reliability layers, voting and self-correction failed to recover accuracy - they only withheld doubtful records. Production systems gain more reliability by replacing generation with deterministic lookups, tracking yield over raw accuracy, and resolving extraction bottlenecks before stacking checks.
Octomind vs Codex CLI: Same Solves, 25% Fewer Tokens Across 150 Runs
octomind.run — GOLD benchmark: Octomind and Codex CLI in 75 same-moment pairs on gpt-6.1-sol. Same solves, 25% fewer tokens, 0.87× the Codex cost, and the harness lesson.
Introducing GIFT-Eval-CTX: An Open-Source Benchmark for Contextual Forecasting
salesforce.com — How well do time-series models handle contextual covariates? Introducing GIFT-Eval-CTX, an open-source evaluation benchmark for next-gen forecasting architectures.
Five AI models tackled 227 pharmacy exam questions. Here's where they struggled
news-medical.net — Researchers evaluated five large language models using 227 infectious disease-related pharmacy examination questions and found that all achieved approximately 90% accuracy, with ChatGPT-4o-mini scoring highest at 91.2%. Accuracy was significantly lower for medicinal chemistry, clinical microbiology, higher-order cognitive questions, and image-containing questions, raising concerns about relying on AI-generated answers without independent verification.
Iterating Towards LLM Reliability with Evaluation Driven Development
www.langchain.com — Dosu uses evaluation driven development and LangSmith to build reliable LLM products at scale, monitor production performance, and iterate with confidence.
Muse Ember (Max) from @AIatMeta just landed in the Code Arena: WebDev as the highest-performing small model - built for…
How to Evaluate Clinical AI Beyond the Demo
www.fiercehealthcare.com — AI demos typically showcase an ideal workflow. | A practical framework for assessing AI governance, evidence, transparency and post-deployment performance in clinical workflows.
In AA-PersonalityBench, models that rank high on sincerity tend to rank low on slop, and models that rank high on creat…
What Is Class Imbalance? Why Accuracy Can Mislead You
www.unite.ai — Class imbalance occurs when one target class is much rarer than another, allowing aggregate metrics and standard training to neglect the cases that may matter most. This guide explains the mechanism, trade-offs, evaluati...
HPC-Bench: A Comprehensive Benchmark for High Performance Computing Codes
hgpu.org — Large Language Models (LLMs) have shown strong capabilities in code generation and reasoning, motivating extensive benchmarks for evaluating functional correctness. However, performance optimizatio…
Dismiss email signup
Subscribe to 
The top AI news in your email delivered each morning. Choose categories