Statistical Analysis for Marketers — The Toolkit Real Operators Use

Most marketing teams use about 5% of the statistical methods available to them, and they often misuse the methods they do use. The honest framework is small: descriptive stats run reporting, inferential stats run testing, regression runs forecasting and attribution, and time-series runs pacing. Each tool has assumptions; violating the assumptions is what generates bad decisions.

Descriptive statistics — the reporting layer

Mean, median, mode, range, variance, standard deviation, percentile. Descriptive stats describe what happened in the data you have. They do not extrapolate. A campaign with a $32 average CAC and a $19 median CAC is telling you the distribution is right-skewed — a few expensive customers are pulling the mean up. The median is closer to the typical experience. Reporting only the mean misleads.

Inferential statistics — the testing layer

Hypothesis testing, p-values, confidence intervals, effect size. The frequentist framework asks: 'If there is no real difference between A and B, how likely is the observed difference?' A p-value of 0.04 means there is a 4% chance of seeing this difference by random chance if there is no real difference.

The most common abuse is peeking — checking the test result before sample-size completion and stopping the test when it looks favorable. This inflates false-positive rates dramatically. Use sequential testing methodology (mSPRT, AGILE) if you need to peek; otherwise commit to the pre-specified sample size.

Regression — the attribution and forecasting layer

• Linear regression (OLS) — predicts a continuous outcome (revenue) from one or more inputs (spend per channel). Assumes linearity, independence, normality of residuals, equal variance. MMM is a multiple linear regression where the inputs are channel spends and the output is revenue.
• Logistic regression — predicts a binary outcome (converted vs not). Used for propensity scoring, lookalike modeling, and click-through prediction.
• Ridge and Lasso regression — variants that handle multicollinearity (when input variables are correlated with each other). Critical for MMM where channel spends often correlate.
• Polynomial regression — for non-linear relationships like diminishing returns curves in MMM.
• Time-series regression (ARIMA, Prophet) — for forecasting where the input is time itself plus seasonality.

Time-series analysis — the pacing layer

Decomposition into trend, seasonality, and residual. ARIMA models for forecasting. Prophet by Meta as a more accessible alternative. Time-series methods are how you forecast Q4 spend three months ahead, detect anomalies in daily reporting, and identify seasonality you should be adjusting bids around.

Sample size and statistical power

The single most impactful statistical concept for marketers: you cannot detect a small effect with a small sample. Power = the probability of detecting an effect if it exists. To detect a 5% lift on a 2% baseline conversion rate at 80% power, you need approximately 70,000 visitors per arm. To detect a 20% lift on the same baseline, you need approximately 4,000 per arm.

Most A/B tests in marketing are underpowered. They run for two weeks, fail to reach significance, and the team declares 'no effect.' What actually happened: the test could never have detected a small-but-real effect because the sample size was too low. Use a power calculator (Evan Miller, Optimizely, VWO have public ones) before launching every test.

Common abuses

P-hacking — running 20 tests, finding the one with p<0.05, and reporting only that one. Bonferroni correction for multiple comparisons is required: divide alpha by the number of tests. Twenty tests at alpha=0.05 should actually use alpha=0.0025.

Simpson's paradox — a trend reverses when data is segmented. A campaign appears to drive lift in aggregate but shows decline in every segment because the aggregate is confounded by segment mix. Always check by segment.

Survivorship bias — analyzing only the customers who converted, ignoring the ones who didn't. Cohort analysis with full retention curves prevents this.

Tool stack

• Excel + Data Analysis ToolPak — adequate for basic regression and t-tests
• Google Sheets + add-ons — XLMiner Analysis for free regression
• Python (pandas + scikit-learn + statsmodels) — production-grade for any analysis
• R — academic gold standard, deeper statistical library than Python
• Stata / SPSS — legacy in social science research, rare in marketing
• JMP / Minitab — for design of experiments (DOE)
• Looker / Tableau / Power BI — for visualization but not for analysis depth

Related guides

• See marketing attribution
• See A/B testing significance
• See regression analysis