63% of AI benchmarks 'solved'? Count only independent runs and it's 24%
AI-generated summary Ethan Mollick published an analysis called SatBench measuring how fast benchmarks are solved. The analysis indicates 63 percent of benchmarks are solved, dropping to 24 percent when counting independent measurements.