www.patronus.ai | Truncated

Saturday 7 November

Benchmarks & Evaluation patronus.ai

Few Word Do Trick: Open Models Cut the Filler

Cracked salt flat in Death Valley, California, stretching to low mountains under a pale sky, in black and white. Photo by Carol M. Highsmith, Library of Congress.

AI-generated summary Patronus.ai measured the fraction of filler words in the reasoning of 12 open models on 20 agentic coding tasks across SWE-bench Verified, SWE-bench Pro and Terminal-Bench 2.1. Patronus.ai says newer releases write fewer filler words, with DeepSeek halving its filler share between April and October builds of its Flash model.

Thursday 5 November

Benchmarks & Evaluation patronus.ai

GlobeBench: A Benchmark for Text-Based World Models

Mount Blackburn, Wrangell Mountains, Alaska. Photo: National Park Service, Alaska Region (public domain)

AI-generated summary Patronus AI introduces GlobeBench, a new benchmark featuring 1,632 tasks designed to evaluate text-based world models across five domains and 19 simulation skills. GPT-6 Astra achieved the highest success rate at 49.57%, while Patronus AI notes that no model solved half of the benchmark tasks.