--- title: Statistical Analysis for Marketers | RGM® url: https://realgrowthmatters.com/learn/measurement/statistical-analysis-for-marketers/ updated: 2026-06-10 source_html: https://realgrowthmatters.com/learn/measurement/statistical-analysis-for-marketers/ --- # Statistical Analysis for Marketers — The Toolkit Real Operators Use Statistical analysis for marketers covers the four operating tools we use daily: descriptive statistics for reporting, inferential statistics for testing, regression for attribution and forecasting, and time-series for pacing. The job is not to be a statistician — it is to know which tool fits which decision and when the tool is being abused. Most marketing teams use about 5% of the statistical methods available to them, and they often misuse the methods they do use. The honest framework is small: descriptive stats run reporting, inferential stats run testing, regression runs forecasting and attribution, and time-series runs pacing. Each tool has assumptions; violating the assumptions is what generates bad decisions. ## Descriptive statistics — the reporting layer Mean, median, mode, range, variance, standard deviation, percentile. Descriptive stats describe what happened in the data you have. They do not extrapolate. A campaign with a $32 average CAC and a $19 median CAC is telling you the distribution is right-skewed — a few expensive customers are pulling the mean up. The median is closer to the typical experience. Reporting only the mean misleads. ## Inferential statistics — the testing layer Hypothesis testing, p-values, confidence intervals, effect size. The frequentist framework asks: 'If there is no real difference between A and B, how likely is the observed difference?' A p-value of 0.04 means there is a 4% chance of seeing this difference by random chance if there is no real difference. The most common abuse is peeking — checking the test result before sample-size completion and stopping the test when it looks favorable. This inflates false-positive rates dramatically. Use sequential testing methodology (mSPRT, AGILE) if you need to peek; otherwise commit to the pre-specified sample size. ## Regression — the attribution and forecasting layer - **Linear regression (OLS)** — predicts a continuous outcome (revenue) from one or more inputs (spend per channel). Assumes linearity, independence, normality of residuals, equal variance. MMM is a multiple linear regression where the inputs are channel spends and the output is revenue. - **Logistic regression** — predicts a binary outcome (converted vs not). Used for propensity scoring, lookalike modeling, and click-through prediction. - **Ridge and Lasso regression** — variants that handle multicollinearity (when input variables are correlated with each other). Critical for MMM where channel spends often correlate. - **Polynomial regression** — for non-linear relationships like diminishing returns curves in MMM. - **Time-series regression (ARIMA, Prophet)** — for forecasting where the input is time itself plus seasonality. ## Time-series analysis — the pacing layer Decomposition into trend, seasonality, and residual. ARIMA models for forecasting. Prophet by Meta as a more accessible alternative. Time-series methods are how you forecast Q4 spend three months ahead, detect anomalies in daily reporting, and identify seasonality you should be adjusting bids around. #### RGM Experts Say Most marketers do not need to know how to derive a t-test from scratch. They need to know that a test with n=200 visitors per arm is almost never powered enough to detect a 5% effect, that p-hacking is real and produces fake wins, and that regression results are only as good as the input data quality. The literacy gap is interpretation, not math. ## Sample size and statistical power The single most impactful statistical concept for marketers: you cannot detect a small effect with a small sample. Power = the probability of detecting an effect if it exists. To detect a 5% lift on a 2% baseline conversion rate at 80% power, you need approximately 70,000 visitors per arm. To detect a 20% lift on the same baseline, you need approximately 4,000 per arm. Most A/B tests in marketing are underpowered. They run for two weeks, fail to reach significance, and the team declares 'no effect.' What actually happened: the test could never have detected a small-but-real effect because the sample size was too low. Use a power calculator (Evan Miller, Optimizely, VWO have public ones) before launching every test. ## Common abuses P-hacking — running 20 tests, finding the one with p<0.05, and reporting only that one. Bonferroni correction for multiple comparisons is required: divide alpha by the number of tests. Twenty tests at alpha=0.05 should actually use alpha=0.0025. Simpson's paradox — a trend reverses when data is segmented. A campaign appears to drive lift in aggregate but shows decline in every segment because the aggregate is confounded by segment mix. Always check by segment. Survivorship bias — analyzing only the customers who converted, ignoring the ones who didn't. Cohort analysis with full retention curves prevents this. ## Tool stack - **Excel + Data Analysis ToolPak** — adequate for basic regression and t-tests - **Google Sheets + add-ons** — XLMiner Analysis for free regression - **Python (pandas + scikit-learn + statsmodels)** — production-grade for any analysis - **R** — academic gold standard, deeper statistical library than Python - **Stata / SPSS** — legacy in social science research, rare in marketing - **JMP / Minitab** — for design of experiments (DOE) - **Looker / Tableau / Power BI** — for visualization but not for analysis depth ## Related guides - See [marketing attribution](/learn/measurement/marketing-attribution-explained/) - See [A/B testing significance](/learn/concepts/ab-testing-statistical-significance/) - See [regression analysis](/learn/measurement/regression-analysis-for-marketing/) ## Sources 1. [1]Evan Miller statistical power tools; statsmodels and scikit-learn documentation; Wasserman, All of Statistics ### Related guides - [Marketing attribution](/learn/measurement/marketing-attribution-explained/) - [A/B testing significance](/learn/concepts/ab-testing-statistical-significance/) - [Regression analysis](/learn/measurement/regression-analysis-for-marketing/) - [MMM Ultimate Guide](/learn/measurement/marketing-mix-modeling-guide/)