Predictive Validity

Predictive validity is the degree to which a metric accurately predicts a future outcome it is supposed to relate to. If a lead score is meant to forecast which leads will convert, its predictive validity is how well high scores actually correspond to conversions — whether the metric's predictions come true. It is a specific type of validity, focused on the criterion of future prediction: a measure has high predictive validity if it reliably forecasts the outcome it should, and low predictive validity if its predictions do not pan out. Predictive validity is established by checking the metric's predictions against what actually happens — does the thing the metric predicts in fact occur as predicted? It is the form of validity most directly tied to a metric's usefulness for forecasting and decisions about the future.

Predictive validity matters because many marketing metrics exist to predict — lead scores predict conversions, propensity models predict churn or purchase, awareness metrics are taken to predict future sales — and a predictive metric is only useful if its predictions are actually accurate. A lead score with poor predictive validity sends sales chasing leads that will not convert and ignoring ones that would; a churn model with poor predictive validity wastes retention effort on the wrong customers. Predictive validity is the quality that determines whether a forecasting metric earns its keep. It is tested empirically — by comparing predictions to outcomes — which makes it one of the more concrete and checkable forms of validity. When a metric is used to predict and act on the future, its predictive validity is the quality that matters most.

Predictive validity is one specific type within the broader concept of validity. Validity in general asks whether a metric measures what it claims; predictive validity narrows that to a particular claim — that the metric predicts a future criterion — and tests it by whether the predictions hold. Other types of validity (content validity, construct validity) ask whether a measure covers and behaves like its concept; predictive (or criterion) validity asks specifically whether it forecasts an outcome. So predictive validity is a focused, empirically checkable form of the umbrella quality of validity, concerned with forecasting rather than with conceptual coverage. A metric can have strong predictive validity for one outcome and weak validity in other senses, which is why predictive validity is named and assessed specifically.

Crucially, predictive validity is about prediction, not causation, and the two must not be confused. A metric can predict an outcome accurately without causing it — it may correlate with the outcome because both share a common cause, or because the metric is a downstream symptom rather than a driver. A weather-vane predicts wind direction without causing it. So a metric with high predictive validity tells you the outcome is likely, but not that intervening on the metric will change the outcome. This distinguishes predictive validity from causal metrics: predictive validity asks 'does this forecast the outcome?', while causation asks 'does changing this change the outcome?'. Confusing the two leads to acting on predictors as if they were levers — trying to move an outcome by manipulating a metric that merely predicts it without causing it. Predictive validity is a forecasting quality, not a causal one.

Establishing predictive validity means testing a metric's predictions against actual outcomes: take the metric's forecast (this lead will convert, this customer will churn), wait for or examine the real outcome, and measure how well predictions matched reality. A metric with strong predictive validity shows that its predictions reliably come true; one with weak predictive validity shows predictions that do not pan out. Good practice validates predictive metrics empirically and re-validates them over time, because predictive relationships can decay as conditions change — a model that predicted well last year may not this year. Predictive validity is earned through demonstrated forecasting accuracy, not assumed from plausibility, and maintained by ongoing checking against outcomes.

The failures are assuming a metric predicts well without testing it against outcomes, treating a predictor as a causal lever (trying to move the outcome by manipulating a metric that only forecasts it), and letting predictive validity decay as conditions change without re-validating. A team that trusts an old lead-scoring model whose predictive relationship has since broken down acts on stale forecasts. The discipline is to establish predictive validity empirically — testing predictions against actual outcomes and re-validating over time — and to use predictive metrics as forecasts, not as causal levers, recognizing predictive validity as the specific, checkable form of validity that determines whether a forecasting metric's predictions can actually be trusted, distinct from both general validity and from causation.

Predictive validity — how well a metric forecasts a future outcome it should relate to — is the specific, empirically tested form of validity that determines whether a forecasting metric's predictions can be trusted.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where predictive validity is a core concern: