Craft
The Evidence Section: What Counts as Proof
A citation in a PRD says nothing about its quality. What counts as proof a problem is real, and how to make sample, recency and source visible.
Evidence: customer interviews, support tickets.
That line appears in a lot of specs, and it proves almost nothing. It tells you a source exists. It does not tell you who was interviewed, how many, when, what they actually said, or whether the ticket count is three or three hundred.
This is where the most expensive spec failure gets in. The well-formed wrong thing, a coherent, fully specified spec built on a false premise, rests on an evidence section that looked fine. Nobody lied. The citation was real. Its quality was invisible, so nobody weighed it.
The fix is not more evidence. It is making the quality of the evidence you already have visible on the page, so a reader can weigh it in ten seconds instead of taking it on trust.
What counts as proof that a problem is real
Evidence for a problem is anything that would still be true if you had never had the idea. That rules out more than it sounds like.
Roughly from strongest to weakest:
| Kind | Example | What it can show | What it cannot |
|---|---|---|---|
| Behaviour | Workarounds, repeated manual steps, abandoned flows in the logs | The problem happens, and how often | Why, or whether people mind |
| Cost | Time lost, money spent on a substitute, churn reasons | It matters enough to pay for | Whether your fix recovers the cost |
| Unprompted complaint | Tickets, reviews, a customer raising it on a sales call | People notice it | How common it is among people who never complain |
| Prompted answer | Responses to an interview or survey question about the problem | How people describe it | What they do when nobody is asking |
| Request | "Can you add X?" | Someone imagined a solution | That X addresses their problem |
| Opinion | "Customers want this," from inside the company | Someone believes it | Anything about customers |
Two things about the table. It is an ordering of kinds, not a law: one interview in which someone walks you through their spreadsheet workaround beats a page of survey ticks. And the bottom two rows are not worthless. They are leads. The trouble starts when they are written up in the same voice as the top rows.
Why the citation hides the quality
A reviewer reading Evidence: customer interviews has two options: trust it, or go and dig. Most trust it, because digging means asking a colleague to show their homework in public.
An automated reviewer has the same blind spot for a different reason. It can see that research was cited. It cannot see that the research was six interviews with your friendliest customers. The citation is in the document; its quality is not.
AI-drafted specs make this worse, because a model with no evidence will still write "users frequently report frustration" in a confident voice. Checking a generated draft starts with asking for the source of every sentence like that.
So the job of the evidence section is not to prove the problem. It is to let the reader judge how well it has been proven.
Five things to write next to every piece of evidence
Each fits in a clause. Together they turn a citation into something a reader can weigh.
Who. Which users, in which segment, in which role. "Admins at accounts with over 50 seats" and "users" are different evidence.
How many. The count, and the denominator where there is one. "4 of 6 interviewees" is honest. "Most users we spoke to" hides that it was six.
When. The date range. A problem documented before a redesign may not survive it. Evidence has a shelf life, and the date lets a reader decide whether it has expired.
How they were found. Did they volunteer, or did you pick them? Customers who asked for the feature, customers on a churn list and a random sample of active accounts will tell you different things about the same problem. Write down which group you had.
Said or inferred. The most important one. Is the sentence in the spec what they said, or what the team concluded from what they said? "Three admins described exporting to a spreadsheet every Monday to reconcile seats" is an observation. "Admins need automated seat reconciliation" is an inference. Both belong in the spec. They should not be the same sentence.
Before and after
An illustration. The evidence section as it often arrives:
Evidence. Customers have told us that seat management is painful (interviews, support tickets). Sales says it comes up in renewals.
Every claim in that paragraph might be true. None of it can be weighed.
The same evidence, with its quality showing:
Evidence.
- Support tickets: 23 tickets tagged "seats" from June to August, from 17 accounts; 14 of the 17 have over 50 seats. Behaviour, unprompted.
- Interviews: 6 admins, all from accounts that had filed a seat ticket, interviewed in August. 4 described a weekly manual export to reconcile seats; 2 called it a minor annoyance. Summarised from their words; notes linked.
- Sales: seats mentioned in the notes for 3 of 40 Q3 renewals. Unprompted, but recorded by the rep rather than the customer.
- Inference: large accounts spend recurring admin time on reconciliation. Not yet measured: how long it takes, or whether smaller accounts have the problem and never report it.
The second version is a few lines longer. It is also the one a reviewer can argue with. They can see that every interviewee was pre-selected for having complained, that the sales signal is three renewals, and that the cost has not been measured. That is not a weakness in the document. It is the document doing its job.
Write down what counts against
An evidence section that lists only support has been curated, whether or not anyone meant to curate it. Add a line for what you looked for and did not find, or found pointing the other way: the two interviewees who called it minor, the segment with no tickets at all, the usage data showing the current workaround is rarely touched.
This is the cheapest place to catch the well-formed wrong thing, because it makes someone look for the reason the premise might be false before the spec is built on top of it.
Cite the parts of a study, not the link
The same discipline applies to outside research. A study is evidence with the same attributes: who was studied, how many, when, how they were chosen, and what was measured as distinct from what the authors concluded.
Our briefing on how evidence evolves has a clean example. A 2014 experiment with 15 students concluded that passive voice in requirements has a strong effect on missing associations in domain models. Re-examined one step at a time, the best-supported estimate became more missed associations in 39% of cases, the same number in 47% and fewer in 14%. A spec citing the first paper as "passive voice makes requirements worse" would have been citing something real and overstating it.
So when you cite a study, write the population, the sample and the measure, and say whether anyone has replicated it. One clause each.
Reviewing an evidence section
Four questions, in order:
- Which of these sentences are observations, and which are conclusions? If you cannot tell, neither can the team building from it.
- Who is missing? A pre-selected sample is fine when it is labelled. The problem is a spec that generalises from the people who asked to the people who did not.
- How old is it? Anything from before the last significant change to this surface deserves a date and a question mark.
- What would change the decision? If the honest answer is "nothing", the evidence section is decoration and the real reason for building lives somewhere else. Write that reason down instead.
An automated pass can check that an evidence section exists and that claims point at a source. Whether the source says what the spec claims is the reviewer's question, asked out loud, before the build. Framing the problem first only helps if the proof underneath it can bear weight.
FAQ
What counts as evidence? Anything that would be true without your idea: behaviour, cost, unprompted complaints. Requests and opinions are leads, and should be labelled as leads.
Why isn't a citation enough? It shows a source exists, not who was asked, how many, when, or whether the claim is theirs or yours.
What should each piece include? Who, how many, when, how they were found, and said versus inferred.
What if it is thin? Say so. Honest thin evidence tells the team how much to bet.