AI Backtest Guardrail Fails: Benchmark Finds Date Prompt Cuts GPT-5 Knowledge Claims 39% to 1%
AI-generated summary Instructing GPT-5 to answer as of a past date reduces the share of answers claiming prior knowledge from 39% to 1%, while its accuracy changes from 76% to 75%. A benchmark called FinDendro posted to arXiv on Nov. 4 by a 15-author team tests safeguards against data leakage using 2,226 dated U.S. facts.