Causal Inference Methods

The analytical frontier for marketing. Correlation vs causation, RCTs, DiD, synthetic control, RD, IV, PSM, CausalImpact.

Why causal inference matters

Marketing teams swim in correlational data: this customer saw an ad and bought. But did the ad cause the purchase, or would the customer have bought anyway? Most marketing measurement systems implicitly assume correlation equals causation, leading to systematic over-attribution.

Causal inference is the toolkit for answering "did our marketing actually cause this?" questions. It's the analytical frontier separating sophisticated programs from naive ones.

Correlation vs causation

• Confounders. A third variable causing both X and Y appears as "X causes Y".
• Selection bias. Treated and untreated groups differ in ways that affect outcome.
• Reverse causation. Y causes X, not X causes Y.
• Random chance. Patterns in noise.
• The mistake: Inferring causation from observational correlation without addressing these.

Randomized controlled trials (A/B tests, incrementality tests)

• Gold standard. Random assignment to treatment vs control eliminates confounders.
• Where feasible, always preferable to quasi-experimental methods.
• Constraints: not all marketing can be randomized; geo-level when user-level not feasible.
• Covered extensively in CRO Experimentation and Attribution & Measurement series.

Difference-in-differences (DiD)

The workhorse for geo holdouts and natural experiments.

• Setup: Treatment and control groups; pre-period and post-period.
• Calculation: (Treatment_post − Treatment_pre) − (Control_post − Control_pre).
• Logic: Removes both time-trend and group-fixed differences.
• Key assumption: Parallel trends in pre-period.
• Use cases: Geo holdouts; policy changes; competitor exits.

Synthetic control

• When you have one or few treatment units and many potential control units.
• Constructs a weighted combination of control units that best matches treatment in pre-period.
• Post-period divergence = causal effect.
• Particularly useful for: state-level policy changes, marketing in single major market.
• Methods: synthcontrol R package, synth Python package.

Regression discontinuity

• When treatment is assigned by crossing a threshold (e.g., loyalty program eligibility at $500 spend).
• Compare units just above and just below threshold — effectively random near cutoff.
• Use cases: tier-based eligibility, score cutoffs.
• Caveat: requires sharp discontinuity in treatment assignment.

Instrumental variables

• When you can't randomize but a third variable (instrument) affects treatment but not outcome directly.
• Example: weather affects retail visits (instrument); visits affect sales (treatment); weather doesn't directly affect online sales (exclusion restriction).
• Complex; requires strong domain knowledge and careful instrument selection.
• Common pitfalls: weak instruments; violated exclusion restrictions.

Propensity score matching

• For observational comparisons; matches treated units to similar untreated units.
• Propensity = probability of receiving treatment, estimated from covariates.
• Matched comparison reduces (but doesn't eliminate) selection bias.
• Use cases: comparing customers who used feature X vs didn't; comparing acquired vs not-acquired similar prospects.
• Caveat: requires good covariates; unobserved confounders remain.

CausalImpact

• Bayesian structural time-series approach (Google, open source).
• Estimates counterfactual: what would have happened without intervention?
• Works with limited control units; learns from pre-period autocorrelation.
• Use cases: campaign launches, single-market interventions, geo holdouts.
• R package and Python implementations.

Advanced playbook

• RCT first. When you can randomize, do. Only use quasi-experimental when you can't.
• Pre-period diagnostic plots. Verify parallel trends for DiD before interpretation.
• Sensitivity analysis. How robust is your causal estimate to assumption violations?
• Triangulation across methods. Same conclusion from multiple methods = stronger evidence.
• Pre-registration for high-stakes causal claims. Document method, data, analysis before observing results.
• Causal DAG documentation. Directed acyclic graph showing assumed causal relationships informs method choice.
• Domain expert consultation. Causal claims require subject-matter knowledge; data alone isn't enough.
• Bayesian methods for uncertainty quantification. Credible intervals on causal estimates.
• Annual incrementality calendar. Schedule causal studies on largest programs.
• Calibration with MMM. Causal estimates inform MMM priors; MMM informs which causal studies to run next.

Common mistakes

• Correlation reported as causation in stakeholder communication.
• RCT skipped when feasible; quasi-experimental relied on unnecessarily.
• DiD parallel trends not checked.
• Synthetic control without good donor pool; misleading counterfactual.
• RD without sharp threshold; weak identification.
• IV with weak instrument; effectively meaningless.
• PSM with poor covariates; selection bias remains.
• Single method without triangulation; over-confidence.
• Causal DAG unstated; assumptions implicit and untestable.
• No sensitivity analysis; brittleness unknown.
• Domain experts not consulted; causal claims naive.
• Point estimates without uncertainty quantification.

Operating checklist

• RCT preferred when feasible; quasi-experimental only when not
• Causal DAG documented per study
• Pre-period diagnostic plots for DiD
• Triangulation across methods for high-stakes claims
• Sensitivity analysis on assumption violations
• Bayesian uncertainty quantification
• Pre-registration for high-stakes causal studies
• Domain expert consultation
• Annual incrementality calendar
• Calibration loop with MMM
• Documentation of method choices and rationale
• Stakeholder communication distinguishes causal from correlational

---

Part of the Marketing Analytics series.