A wrong AI summary changed what people remembered seeing

Most teams now have some version of the same habit. A spec goes out with an AI-generated summary at the top, or a review meeting ends and a bot posts the recap. The usual worry is that the summary might be wrong. A study posted on arXiv on 23 September 2026 tests a less comfortable question: when the summary is wrong, does it change what the people who were there remember?

In this experiment, it did, and knowing the summary came from an AI made no significant difference.

The study

The paper is by Mattea Sim, Yael Eiger and Tadayoshi Kohno of Georgetown University and the University of Washington. A shortened version appears in the proceedings of the AAAI/ACM Conference on AI, Ethics, and Society in October 2026. It has two parts.

Part one: what goes wrong in AI summaries. The authors asked ChatGPT-5.5 and Gemini 2.5 Flash-Lite to summarise two 25-second animated videos of a car hitting a pedestrian — stimuli taken from a classic line of memory research. Each model summarised each video five times, giving 20 summaries. The prompt was deliberately generous: factual, neutral, at least 300 words, with an explicit instruction to include everything important.

Part two: what a wrong summary does to memory. 360 US adults recruited through Prolific took part. Each watched one of the videos, which showed the car at either a stop sign or a yield sign. A day or two later, they read a 21-sentence summary written by ChatGPT. For half of them, the summary named the sign they had seen; for the other half, the wrong one. Separately, half were told the summary was written by AI, half that it was written by a professional human transcriber — though the text was the same. Then they answered memory questions about the video. 328 participants were included in the analysis.

What they found

Every AI summary contained mistakes. All 20 did, with between seven and twenty-one errors each in summaries that averaged 305 words. The errors were also inconsistent: a car turning right in one run turned left in the next, and vehicles or pedestrians appeared that were not there.

Omission was the dominant error. Every summary left something central out. On average, the summaries omitted 51.6% of the 19 central details the researchers had listed. 95% — all but one — left out the most central event of all: the car colliding with the pedestrian. Some dropped the pedestrian entirely.

Other error types were common too: 65% of summaries described at least one central detail inaccurately, and 60% added at least one detail that was not in the video.

A wrong detail in the summary displaced the memory. Of participants who read an accurate summary, 83.6% correctly recalled which sign the car stopped at. Of those who read a summary with the sign changed, 44.8% did. Prior work with the same videos found 80.26% of people recall the sign correctly with no summary at all.

Scepticism did not measurably protect. Among participants who read the misleading summary, 46.3% of those told it was AI-written answered correctly, against 43.4% of those told a human wrote it — no significant difference. Neither trust in AI nor how often participants used AI chatbots changed the effect.

The participants had not simply forgotten the video. 96.6% correctly remembered the accident happened in daytime, a detail the summary never mentioned. They remembered the event; the summary rewrote one part of it.

The authors' framing of what this means is direct:

Though "humans-in-the-loop" are often expected to correct for AI's mistakes, our work suggests human memory can instead be distorted by these mistakes.

The caveats, from the paper itself

The authors list their limits plainly, and they matter for anyone tempted to stretch the result.

• Narrow material. Two short, simple videos of a video-game accident. The authors say this limits generalisation to more realistic events, and that their analysis of model output used a single prompt and a small number of summaries.
• One kind of misinformation. The human experiment tested one changed critical detail. It did not test what the most common AI error — omission — does to memory, though the authors point to older human-written research suggesting omissions degrade recall too.
• A curated summary. Because ChatGPT often left the accident out, a researcher re-prompted it until a summary included the collision, selected one of the higher-quality outputs, and made minor edits so it fit both videos. If anything, participants read a cleaner summary than a typical first try.
• A lab, not a workplace. The design follows standard memory-research procedure. The authors call for studies in real high-stakes settings.
• Video, not text or audio. Many real summaries are made from transcripts. The authors expect those could be less accurate, but did not test it.

They also treat the AI-versus-human result with caution, since it rests on a non-significant difference. Absence of a protective effect from scepticism is suggestive, not proven.

Why this matters for reviewers

The study is about eyewitness memory, and its authors are thinking about police reports, not product teams. The translation to spec review is ours. But the two halves of the result line up uncomfortably well with how pre-reads and recaps now work.

An AI pre-read can omit the thing the review is for. The summaries in this study missed the collision — the one event that mattered — 95% of the time, under a prompt that explicitly asked for everything important. A reviewer who reads an AI summary of a spec instead of the spec is reviewing whatever the summary kept. If the open question, the risky assumption or the decision you need from them is the part that got dropped, the review will be confident and beside the point.

An AI recap of a review can rewrite what was decided. The memory result is the sharper one. People who had seen the event themselves, and who were told the summary was AI-written, still more often than not remembered the summary's version. Applied to a review meeting, the risk is not just a wrong recap in the channel. It is a room of attendees who each half-remember the wrong decision and agree with each other. We have argued for sending no separate recap at all and keeping decisions in the doc; this is a further reason, and a stronger one when the recap is generated.

Three precautions follow, none of which require giving up the tools:

1. Write the top-of-doc summary yourself. The author knows which detail is the collision. Put the ask and the open questions there, and link each line to the section it summarises.
2. Check a generated recap against the doc before it goes out. The study suggests readers' own memory will not reliably correct it later, whatever they think of AI.
3. Keep one record. If the decision lives in the spec, a wrong summary is a nuisance. If the summary is the only place it was written down, the summary becomes the decision — the failure described in where product decisions go to die.

FAQ

What was tested? Error rates in AI video summaries, and whether one misleading detail in a summary changes what viewers remember.

The headline result? Correct recall fell from 83.6% after an accurate summary to 44.8% after a misleading one.

Did knowing it was AI help? No significant difference, and trust in AI did not moderate it.

Biggest caveat? Two short animated videos and one manipulated detail; the step to meetings and specs is an inference, not a finding.