Thinkr

Craft

Your Metric Is Missing Its Baseline

"Increase activation to 40%" — from what? The most common defect in a PRD metric, why it happens, and the three honest ways to handle a number you do not have.

Galang Aulia · 4 min read
Craft

"Increase activation to 40%."

From what?

It is the single most common defect in a PRD metric, and it survives review constantly because the sentence looks complete. There is a number, a direction, and a verb. What is missing is the only thing that would let anybody evaluate it.

A target with no baseline is not a measure. It is a wish with a decimal point.

Why it happens

Not laziness. Three structural reasons, and recognising which one you are in changes the fix.

The baseline is genuinely unknown and finding it is work. The event is not instrumented, or it is instrumented differently from how you would need to count it. Getting the number is a small project, and the spec is due Thursday.

The number reads better without context. "Increase activation to 40%" sounds ambitious. "Increase activation from 37% to 40%" sounds like a rounding error — which it might be, and that is precisely the information the baseline was carrying.

The template asked for a target. Most PRD templates have a field called Success metric and no field called Current value. People fill in what they are asked for.

What it costs

You cannot tell success from noise. If activation lands at 41%, did you move it four points or did it fluctuate four points on its own last quarter too? Without a baseline — and ideally some sense of its normal variance — there is no answer.

You cannot size the opportunity. A jump from 12% to 40% is a different project, with different staffing, from 37% to 40%. Reviewers are being asked to approve one without knowing which.

Review cannot evaluate it. This is the quiet cost. A reviewer who cannot assess whether the target is plausible will not challenge it, so the metric section passes untouched — not because it was good, but because it was unevaluable.

Afterwards, the number gets interpreted by whoever is most invested. With no prior commitment about where you started, the result is argued rather than read.

The three honest options

1 · Measure it first. The best answer when the data exists. Pull the number, put it in the spec, set the target against it. Often twenty minutes.

2 · Say it is unknown and make it the first task. "Baseline unknown. We will instrument activation and set a target after two weeks of data." Completely legitimate, and more useful to a reviewer than a confident figure resting on nothing. It also converts a hidden problem into a scheduled one.

3 · Use a relative target with a defined window. "Reduce median time-to-first-critique by 25% against the trailing 30-day median at launch." The baseline is defined by a rule rather than a number — fine, provided the rule is written down before launch rather than chosen afterwards.

What is not on the list is picking a plausible number so the field is not empty. That is the option that costs you credibility on every other number in the document.

Baselines that lie

Having a baseline is not the same as having a good one. Three ways a present baseline still misleads:

Wrong population. The baseline comes from all users; the target will be measured on users who reach the feature. Those are different denominators, and the gap between them can exceed the effect you are trying to detect.

Wrong period. A baseline taken during a launch spike, a holiday, or an outage week. Seasonality alone will produce or erase most product effects if you let it pick your comparison period.

Wrong counting rules. The most common and the hardest to catch, because both numbers are called "activation." The baseline counts anyone who logged in twice; the target will count anyone who completed a first run. Both are defensible definitions. Comparing them measures nothing.

That last one is why counting rules belong next to the metric — a baseline is only a baseline if it was computed the way the target will be.

Net-new features

The easy case: the baseline is zero, and nobody argues.

But zero is only the starting number, and the rest of the metric is where new-feature targets usually go wrong. "25% adoption" still needs a denominator (all workspaces? workspaces created after launch? workspaces with more than one member?) and a window (30 days from GA? ever?).

A new feature makes the baseline free and changes nothing else. The specificity still has to be written.

In review

When you see a bare target, one question resolves it:

What is that number today?

Three possible answers, all useful. "37%" — good, put it in the spec. "I don't know" — good, now it is a known gap with an owner instead of a hidden one. "We can't measure it yet" — best of all, because you just found a dependency with a lead time, before the build rather than during the readout.

The question takes five seconds and it is the highest-yield thing you can ask about a metrics section. It is also the one most reviewers skip, because a target with a number in it looks like it has already been thought about.

FAQ

What is a baseline? The current value of the number you intend to move, measured the way you will measure the target.

What if we don't have it? Say so and make establishing it the first task. An honest unknown beats an invented figure.

Do net-new features need one? The baseline is zero, but the denominator and window still have to be written.

Can a baseline be wrong? Yes — wrong population, wrong period, or wrong counting rules. The third is hardest to spot.

New posts and release notes. No spam, unsubscribe anytime.

Stop shipping foggy PRDs.
Start the critique loop.

Three minutes to sign up. No credit card. Cancel by closing the tab.

Start freeSee pricing