Why your NPS number is lying to you
NPS is the most influential metric in B2B software and one of the easiest to lie with. The lie isn't usually deliberate — it's structural, baked into how the metric is computed and where it gets sampled. Most NPS dashboards are reading their own noise.
What NPS actually computes
You know the formula but it's worth saying out loud: NPS = % promoters (9–10) − % detractors (0–6). Passives (7–8) don't count toward the number at all.
That formula does two things you might not have noticed:
- It is non-linear. A user moving from 7 to 8 doesn't change the metric. A user moving from 6 to 7 changes it by one detractor's worth. A user moving from 8 to 9 changes it by one promoter's worth. Two of the three boundaries on the 0–10 scale matter; the rest don't.
- It is discontinuous. A 6.99 and a 7.00 are categorically different inputs to the metric, even though no human distinguishes between them on the survey form.
These properties make NPS extremely sensitive to sampling. If your sampling shifts even slightly — different survey timing, different population, different copy — the number can move several points without anything changing about the underlying customer experience.
The sampling lies
Five common sampling sins, all of them seen in the wild:
-
Surveying after success events only. "We asked NPS after their first big win." Sure, and your number is correspondingly high. The customers who quit before the first big win are excluded entirely. NPS computed only over surviving users is not a measure of NPS — it's a measure of how good your retention already is.
-
In-app NPS surveys to active users. Same problem. The most disengaged users haven't logged in for weeks, so they're not in the sample. The number you get is the NPS of users who like you enough to be in your product. Of course it's positive.
-
Email NPS to all users, but with a low response rate. The users who respond are non-representative — they're either passionate (positive) or angry (negative). Passives skip the survey entirely. Your NPS becomes more polarized than the underlying population, and which direction the polarization tilts depends on whether you've recently shipped something controversial.
-
Surveying right after support resolution. Users who just had a problem fixed feel grateful. The number measures the support team's gratitude factor, not the product.
-
Sampling rolled up across customer types. SMB users and enterprise users have different baselines. Mixing them produces a number that's a weighted average that nobody on your team can interpret.
What to do instead
You don't need to throw out NPS — you need to read it carefully:
- Sample randomly across all users, not just active ones, on a fixed cadence (quarterly is fine). The number will be lower than your in-app number. That's correct.
- Cohort the result by tenure, plan, and segment. A single NPS number for the whole company is a vanity metric.
- Plot trend, not level. The absolute NPS number is full of methodology bias and not comparable to your competitors'. Your own quarter-over-quarter change is the only signal that's interpretable.
- Read the verbatims, not the score. The free-text response is the actual data. The score is a sorting key for the verbatims.
The board-deck problem
The reason teams keep reporting a single rolled-up NPS number is that boards and executives like one number. It fits on a slide. The fix is editorial, not technical: ship the number with the cohort breakdown alongside it, every time. Train the room to read it as a distribution, not a level. The team that does this looks more rigorous, not less.
NPS is a perfectly fine metric handled badly by most of the teams using it. Read it like a survey scientist would, and it's useful. Read it like a sales-side dashboard, and it's a story you're telling yourself.