On the same 100 ReportBench prompts, the match rate is 'the proportion of statements that are semantically consistent with their cited sources': gpt-4o identifies every statement with an explicit citation link, the cited web page is scraped, gpt-4o locates the passage most relevant to the statement and then gives a consistency verdict, and the verdicts are aggregated per report. Table 1 prints 78.87 percent for OpenAI Deep Research over 88.2 cited statements per report, 31.43 percent for openai-o3 with search over 16.16 cited statements per report, 72.94 percent for Gemini Deep Research, 73.67 percent for claude4-sonnet, 59.24 percent for gemini-2.5-pro and 44.88 percent for gemini-2.5-flash; no intervals. The base models were given SerpAPI search and Firecrawl reading with at most five tool calls per instance and were made to cite in URL form. This is a per-cited-statement support rate as judged by gpt-4o against the cited page; it is not a reference-list match against the survey's ground-truth bibliography (that is the separate precision/recall column, 0.385 / 0.033 for Deep Research) and not a human verdict.

Same collection as the factual-accuracy figure: reports gathered by the five ByteDance BandAI authors from the OpenAI and Gemini web interfaces between July 14 and July 25 (2025) and from batch runs of the base models with search and link-reading tools. gpt-4o performs statement extraction, supporting-passage extraction and consistency verification in three separate steps, which the authors present as more interpretable than a single LLM-as-a-judge score. No agreement figure between gpt-4o and human raters is printed; the counter-check consists of manual audits of individual cases (arXiv:2407.15186 and arXiv:2009.12619 test items) that surface statement and citation hallucinations, and the remark that intermediate outputs are retained for optional human inspection.

Falls if a human re-judging of a sample of the stored cited statements finds the OpenAI Deep Research consistency rate materially different from 78.87 percent, or if rerunning the o3 baseline without the five-tool-call cap closes most of the gap to 31.43 percent. Query to run: from the ReportBench repository, take the per-statement consistency verdicts for OpenAI Deep Research and openai-o3, hand-check 200 cited statements from each, and recompute both match rates alongside a rerun of o3 with the tool-call cap lifted.

The Deep Research wrapper makes o3's citations two and a half times more reliable. Too coarse: the o3 baseline was boxed in with five tool calls per report and a foreign URL citation format, wrote five times fewer cited statements, and both rates are gpt-4o verdicts on a scraped page with no printed human agreement; the gap shows the product pipeline grounding cited statements better under this setup, not how much of it the wrapper as such contributes.

https://arxiv.org/pdf/2508.15804v1 p. 8, Section 3.4 Model-Level Comparative Analysis (sentence) and p. 7, Table 1 (Match Rate 78.87% vs 31.43%) | 2025-08-14 · arXiv 2508.15804 · Li, Zeng, Cheng, Ma, Jia, ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks

Proposed by attacker-grok-4.7 in the completeness attack of 2026-09-23 against tradeoff:supports: The cited-statement match rate is 78.87 percent for OpenAI Deep Research against 31.43 percent for o3 with search tools on the same task, so on one benchmark the deep-research wrapper grounds cited statements far better than the base model. Owner's reading of the bearing: It supports the tradeoff as one benchmark data point: cited statements of the deep-research product are judged consistent with their sources far more often than those of the base model with tools, but the base-model condition is handicapped (five tool calls, imposed URL citing) and the judge is unvalidated gpt-4o, so it supports the direction, not the size of the effect.