---
title: Bayesian Decision Making — RGM Training
url: https://realgrowthmatters.com/training/marketing-analytics/bayesian-decision-making/
updated: 2026-06-10
source_html: https://realgrowthmatters.com/training/marketing-analytics/bayesian-decision-making/
---

[Home](../../../index.html) › [Training](../../index.html) › [Marketing Analytics](../index.html) › Bayesian Decision Making

RGM° · Training

# Bayesian Decision Making

Why Bayesian is gaining adoption. Mental models vs frequentist, priors, interpretation, sequential testing, MMM, and decision frameworks.

### What you will learn

1. [Why Bayesian methods are gaining marketing adoption](#why)
2. [Bayesian vs frequentist mental models](#vs)
3. [Priors and where they come from](#priors)
4. [Bayesian interpretation: more intuitive for stakeholders](#interpretation)
5. [Bayesian sequential testing](#sequential)
6. [Bayesian MMM](#mmm)
7. [From probability to decision](#decision)
8. [Tools and frameworks](#tools)
9. [Advanced playbook](#advanced)
10. [Common mistakes](#mistakes)
11. [Operating checklist](#checklist)

## Why Bayesian is gaining adoption

Frequentist statistics produces outputs like p-values that stakeholders misinterpret. Bayesian methods produce outputs like "87% probability variant beats control with expected lift 8–15%" that align more naturally with decision-making.

Beyond communication, Bayesian methods handle sequential testing, prior knowledge integration, and small samples better than frequentist. Modern tools (Statsig, Eppo, Meta Robyn) increasingly default to Bayesian approaches.

## Bayesian vs frequentist

| Aspect | Frequentist | Bayesian |
| --- | --- | --- |
| Probability concept | Long-run frequency | Degree of belief |
| Parameters | Fixed unknowns | Probability distributions |
| Output | P-values, confidence intervals | Posterior distributions, credible intervals |
| Interpretability | Counter-intuitive | More natural |
| Prior knowledge | Ignored | Explicitly incorporated |
| Sequential testing | Requires adjustments | Naturally valid |
| Small samples | Conservative | Better with informative priors |

## Priors

- **Prior:** What you believe before seeing the data.
- **Likelihood:** How likely is the data given parameter values?
- **Posterior:** Updated belief after seeing data; combines prior and likelihood.

### Where priors come from

- **Historical data.** Past similar tests provide informative priors.
- **Industry benchmarks.** Conversion rates in your category.
- **Expert judgment.** Domain expertise quantified.
- **Weakly informative (default).** Light prior assumptions when knowledge is limited.
- **Non-informative / uniform.** No prior knowledge assumed.

### Common concerns about priors

- **Subjectivity.** Different priors produce different posteriors.
- **Solution:** Sensitivity analysis with multiple priors; report range.
- **Solution:** Use weakly informative defaults; let data dominate posterior.
- **Strong priors:** Use when data is limited and prior knowledge is reliable.

## Bayesian interpretation

Common Bayesian outputs:

- **Probability that variant beats control.** "87% probability of positive lift."
- **Expected lift with credible interval.** "Expected 5.2% lift, 95% credible interval 1.8–9.1%."
- **Probability of meeting threshold.** "72% probability of >3% lift."
- **Posterior distribution.** Full distribution of plausible effect sizes.

## Bayesian sequential testing

- Probability of effect updates naturally as data accumulates.
- No false-positive inflation from sequential checking.
- Allows early stopping when probability crosses decision threshold.
- Allows continuation when results are ambiguous.
- Modern tools (Statsig, Eppo) implement this directly.

## Bayesian MMM

- Modern MMM is largely Bayesian.
- Priors set on coefficients based on historical knowledge or experiments.
- Posterior distributions quantify uncertainty in channel contribution.
- Calibration with incrementality experiments straightforward.
- Tools: Meta Robyn, Google LightweightMMM, PyMC, Stan.

## From probability to decision

- **Loss function thinking.** What's the cost of shipping a non-winner vs not shipping a winner?
- **Asymmetric decisions.** If the downside of shipping a bad variant is small (easy to revert), ship at lower probability threshold.
- **Expected value calculations.** Probability × impact; informs investment decisions.
- **Risk-adjusted decisions.** Bayesian credible intervals enable proper risk weighting.

## Tools and frameworks

- **Experimentation platforms.** Statsig, Eppo (Bayesian by default); Optimizely Stats Engine (always-valid p-values, Bayesian-influenced).
- **MMM platforms.** Meta Robyn, Google LightweightMMM, Uber Orbit, Recast, Haus.
- **General-purpose:** PyMC, Stan, NumPyro (Python); brms, rstanarm (R).
- **Cloud:** Google BigQuery ML, AWS SageMaker.

## Advanced playbook

- **Weakly informative defaults.** Start with weak priors; let data drive.
- **Prior sensitivity analysis.** Multiple priors; report range; transparent.
- **Decision thresholds documented.** Probability thresholds for "ship" vs "kill" decisions.
- **Bayesian sequential by default.** Modern experimentation tooling enables this; use it.
- **MMM with experiment-informed priors.** The Bayesian-experiment hybrid is best-in-class.
- **Credible intervals over confidence intervals in reporting.** More natural for stakeholders.
- **Decision-theoretic framing.** Loss functions; expected value calculations.
- **Calibration with reality.** Compare Bayesian predictions to actual outcomes over time.
- **Tooling investment.** PyMC, Stan, NumPyro for in-house teams; vendor tools for less custom needs.
- **Methodology transparency.** Stakeholders understand what's Bayesian and why.

## Common mistakes

- Subjective priors masquerading as objective; sensitivity not tested.
- Bayesian outputs misinterpreted as frequentist p-values.
- Strong priors with limited data; posterior driven by prior.
- Weakly informative priors when strong knowledge exists; informative priors would improve.
- No decision thresholds documented; ambiguous outcomes.
- Posterior distributions reported without probability summaries; stakeholders confused.
- Bayesian MMM without experiment calibration; observational.
- Frequentist sequential adjustments applied to Bayesian methods unnecessarily.
- Tool choice without methodology understanding; opaque outputs.
- No calibration check; predictions never compared to outcomes.
- Bayesian framing in stakeholder communication without explanation.
- Treating Bayesian as "always better" without considering use case.

## Operating checklist

- Bayesian or frequentist choice documented per use case
- Priors documented and sensitivity-tested
- Bayesian sequential testing for experimentation
- MMM with experiment-informed Bayesian priors
- Decision thresholds documented (probability + lift)
- Credible intervals in reporting
- Decision-theoretic framing for stakeholders
- Calibration tracking (predictions vs outcomes)
- Tooling chosen and methodology understood
- Stakeholder education on Bayesian interpretation
- Documentation of prior choices and rationale
- Annual methodology review

## Sources and further reading

- Andrew Gelman, "Bayesian Data Analysis" (textbook)
- Andrew Gelman blog
- Statsig and Eppo Bayesian methodology articles
- Meta Robyn open-source Bayesian MMM
- Google LightweightMMM Bayesian framework
- PyMC, Stan, NumPyro documentation
- Allen Downey, "Think Bayes"
- Richard McElreath, "Statistical Rethinking"
- Frank Harrell — Bayesian methodology
- Cassie Kozyrkov decision intelligence writing
- Booking.com Bayesian experimentation research
- Bayesian Marketing community

---

Part of the [Marketing Analytics](../index.html) series.