Hallucination checkers scored near random on invented claims in peer reviews

An AI-drafted document fails in a particular way. It is mostly right, fluent throughout, and wrong in a few specific sentences that sound exactly like the rest. A benchmark posted to arXiv on 3 September 2026 measures how well current tools catch those sentences in one kind of document, the scientific peer review. The answer is: not well across a whole document, and better when the check goes sentence by sentence against the source.

The study

Researchers at National Yang Ming Chiao Tung University in Taiwan built HalluPeer from public OpenReview records for ICLR and NeurIPS. They frame the problem as source-grounded verification: a claim in a review is a hallucination if the paper it reviews does not support it or contradicts it. Tone and opinion are deliberately out of scope.

The construction:

• A taxonomy of 265 review-specific error types, grown from nine coarse categories such as Number, Entity, Attribution Failure, Overgeneralization, Hyperbole and Temporal. It was expanded by LLMs and finalised by human experts.
• Injected errors. An LLM editor rewrote single sentences of real reviews to contain one of those errors while keeping the style, and a second check discarded rewrites that did not change the meaning. The authors' examples of the target failure: asserting a missing baseline, misreporting results, fabricating assumptions.
• Scale. Experiments covered 12K papers and 38K reviews.

Verifiers were tested on three tasks: does this review or sentence contain an invented claim, what type is it, and exactly which words are wrong. The contenders were four specialised hallucination verifiers, seven prompted general-purpose LLMs including GPT-5.2, a retrieval-augmented LLM judge, and models fine-tuned on the benchmark's training data.

What it found

Off-the-shelf verifiers were near random on whole reviews. All four specialised verifiers scored a Matthews correlation of 0.03 or lower at the review level, where zero is chance. The authors put this down to the gap between the general fact-checking data they were trained on and long technical papers that need reasoning across sections.

Sentence by sentence beat whole-document. Prompted frontier models were competitive on single sentences but consistently weaker on whole reviews. GPT-5.2 reached a Matthews correlation of 0.56 on sentences and 0.21 on reviews. The authors attribute this to difficulty keeping verification consistent over a long context. When they turn to classifying and locating the errors, they add that the invented claim is a small span inside largely correct text.

hallucinated content is often sparse and embedded within otherwise correct scientific critique

Numbers and names were the easy part. Asked to classify the type of an invented claim, prompted models did relatively well on categories with explicit lexical cues, such as wrong numbers and wrong entities. Context-based meaning errors, hyperbole and temporal errors stayed hard, with several models near zero at the review level. The best review-level macro-F1 for any prompted model was 0.19.

Specialised training changed the picture, on this data. A fine-tuned Qwen3-32B reached F1 of 0.90 on reviews and 0.91 on sentences. Detectors trained on one conference transferred to the other, and results held when a different model did the injecting.

On real reviews, it found the errors and flagged plenty more. The authors annotated 1,161 genuine NeurIPS 2024 reviews and found 20 real hallucinations. A detector trained only on synthetic data caught all 20, at a false positive rate of 22.1%. Inspecting flagged real reviews surfaced a further complication: some apparent hallucinations were reviewers faithfully quoting exaggerated claims from the paper itself.

The caveats, from the paper

• Synthetic errors. The injected claims are localised single-sentence edits. The authors say real errors may involve subtler, drift-based reasoning failures that are hard to simulate this way.
• One field. Computer science conferences on OpenReview only; the authors say the taxonomy and detectors may not generalise to other disciplines without adaptation.
• Factual errors only. Unfair novelty judgments, tone and other subjective failures are excluded.
• Model-shaped taxonomy. Because LLMs proposed the categories, the hierarchy may inherit their biases; the authors call it a model-centric view rather than a canonical standard.
• Selection bias. Only reviews that aligned with the meta-review were used as bases for injection, which favours reviews whose points mattered to the final decision.

And two of ours. The fine-tuned detectors were trained on data built the same way as the test set; cross-venue and cross-generator checks help, but the authentic-review check rests on 20 real errors. And a peer review is not a PRD. The transfer is in the method, not the numbers.

Why it matters if you review AI-drafted specs

A peer review is a document about a source document, and so is a PRD drafted by AI from your research notes, metrics and past decisions. The failure HalluPeer measures, a fluent claim the source does not support, is the failure that matters most in reviewing an AI-drafted PRD.

Verify claims, not impressions. The benchmark's framing is the useful part: a claim is wrong if the source does not support it. For an AI-drafted spec, list the factual claims and trace each to an input. A baseline, a user quote, a "customers keep asking for": if it is not in what you gave the model, it was invented. Our post on evaluating an AI PRD generator makes the same point about counting invented numbers.

Read at sentence level. Whole-review judgments were where every prompted model was weakest, because one bad sentence hides in a page of good ones. A reader skimming a ten-page draft for overall plausibility has the same problem, and an invented baseline looks like a decision.

Check numbers first, but do not stop there. Wrong numbers and names were the easiest category to identify. Overstatement, misplaced context and errors of timing were the hardest, and overstatement is the shape of many claims a spec makes about users, in words like "significant demand", "critical" or "customers love". Those need the most attention, not the least.

Do not hand the check to a general assistant and call it done. Off-the-shelf checking was weak here, and only specialised training on this exact task did well. That fits what a requirements benchmark found about off-the-shelf models reviewing requirements: the judgment-heavy errors are the ones missed.

Tracing to the source catches invention, not inherited error. Some flagged claims were faithfully copied from the paper being reviewed. An AI draft can do the same with your research doc. If the input overstated something, a perfectly grounded draft will overstate it too.

FAQ

What was measured? Whether verifiers can detect, classify and locate false claims injected into real peer reviews, across 12K papers and 38K reviews.

How did checkers do? Off-the-shelf verifiers were near random on whole reviews; prompted LLMs did better on single sentences than whole reviews.

Hardest errors? Exaggeration, misplaced context and timing, rather than numbers and names.

What changes in review? Trace claims one by one to their source, and give overstatement and context-dependent claims the most scrutiny.