A/B Test Budget Calculator
How much ad spend does it take to get a trustworthy answer — and is the lift even provable in your market? Set your baseline, the lift you want to detect, your cost per click, and your obtainable market. This works backward from statistical significance to the sample, the budget, and a feasibility verdict.
An A/B test only means something with enough sample to detect a real difference — and your market caps how much sample you can ever get. This calculator finds the sample per variation with a two-proportion z-test, corrects it for your finite obtainable market, and returns the budget, the feasibility verdict, and the smallest lift your market can ever prove — so you never call a winner on noise, kill one before it proves itself, or chase a lift no budget could resolve.
Start with what you can reach
Your serviceable obtainable market (SOM) — the audience you can realistically reach for this test — sets a hard ceiling on what any experiment can prove. So the calculator starts here, then bounds the statistics to it. TAM and SAM set the context; SOM does the work.
Enter audience counts — people, sessions, or accounts. Thinking in revenue? Divide each market by your average revenue per customer. Only SOM bounds the test.
A/B test inputs and result
| Confidence | Sample / variant | Budget | Verdict |
|---|
How to use this calculator
- Size the audience you can actually reach.Enter TAM and SAM for context, then set SOM to the number of people, sessions, or accounts you can put through this test in its window. SOM is the only input that limits the math, so be honest about it.
- Enter the control’s baseline rate.Use the metric you’re testing — conversion rate for a page, click-through rate for an ad. A 5% baseline behaves very differently from a 0.5% one.
- Name the smallest lift worth detecting (MDE).Be deliberate. A bold 30–50% MDE is cheap to prove; a subtle 3–5% MDE can cost 15–20× more. Pick the smallest change that would actually change a decision.
- Add cost and test design.Set cost per unit (CPC, CPM÷1000, or CPA), the number of variations, and your confidence and power. Leave them at 95% / 80% unless you have a reason not to.
- Read the verdict, then export.Check the budget, the percentage of SOM consumed, and the feasibility verdict. Use the confidence table to see the trade-off, then copy a share link, download the CSV, or print a one-page PDF for your test plan.
RGM Expert Says
We reach for this calculator before a single dollar is committed to a test, in the planning meeting where someone says “let’s just run it and see.” That sentence is where money quietly leaks. The tool turns a vague ambition into a number you can defend: this much sample, this much spend, this likely an answer — or, just as valuable, a clear “not in this market.”
It earns its keep most on three jobs. Greenlighting a roadmap of tests: we run every proposed experiment through it and immediately see which ones are affordable and which are fantasies, so the quarter’s testing calendar is built from things that can actually conclude. Pricing a single high-stakes test — a checkout redesign, a new pricing page — where the MDE is small and the cost of a wrong call is large. And settling the “is it significant yet” argument by showing, up front, how much traffic significance was always going to take.
The way to get the most from it is to be ruthless about two inputs. First, set SOM to what you can truly reach in the window, not your whole list — a 200,000-person database that only sends 18,000 sessions a month to the tested page has a SOM near 18,000, not 200,000. Second, treat MDE as a business decision, not a statistics knob: the smallest lift that would change what you do next. Do those two things and the budget the tool returns is one you can take to a CFO. When the verdict comes back “not provable,” that is not the tool failing — it is the tool saving you a quarter of spend on a test that could never have produced a trustworthy answer.
How it works
You can’t budget an experiment by gut. Determining the spend for a statistically significant test is one of the highest-leverage steps in growth marketing: get it wrong and you burn budget on false positives, or kill a winning variation before it has a chance to prove itself. So you work backward — first the sample size needed to detect a true difference, then the spend to buy that sample.
First, the sample size N per variation, from the standard two-proportion z-test:
Then — the step most calculators skip — we bound it by your obtainable market with the finite-population correction. With Ng = SOM ÷ variations reachable people per arm:
Inverting the same math at a full census of your market gives the hard floor — the smallest lift you could ever resolve:
Then the spend buys the corrected sample:
- p₁ — baseline rate (e.g. 0.05 for 5%). p₂ — expected new rate = p₁ × (1 + MDE). p̄ — pooled rate = (p₁ + p₂) / 2.
- zα/2 — critical value for significance (1.96 at 95% confidence). zβ — critical value for power (0.84 at 80%).
- Ng — reachable audience per variation (SOM ÷ variations). When the raw sample N exceeds Ng, the lift is unprovable in your market at any budget — the floor is the smallest lift that is.
The two-proportion test and finite-population correction are standard statistics (see Wang & Chow, 2007, worked reference). The market-bounded floor — reading SOM as a finite population and inverting the test to the smallest provable lift — is RGM’s own framing.
Most experiments don’t win — so don’t pay to learn nothing
Experimentation at scale is humbling. In a study of more than 127,000 experiments, Optimizely found only a small fraction move the primary metric — roughly one in eight wins (Optimizely, 2023). Microsoft has reported that only about a third of tested ideas improve the metric they targeted (HBR, 2017). When most tests don’t win, the tests that do run have to be powered well enough to trust — otherwise you are paying real money to generate noise.
The MDE drives almost everything. A bold 50% lift needs a small sample, while a subtle 5% lift can need 15–20× more — which is exactly why naming the effect you care about, up front, is a budgeting decision, not a statistics footnote. Under-fund a test and a random good week looks like a win, so you scale noise. Worse, “peeking” at the result and stopping early inflates the false-positive rate well beyond the 5% you think you’re running at (Georgiev, Analytics-Toolkit).
This is where TAM / SAM / SOM stop being a pitch-deck slide and start being math. A subtle lift on a small obtainable market can demand more sample than the market contains — the test is unwinnable before you spend a dollar. By correcting the sample for your finite SOM and inverting it to a floor, the calculator tells you the smallest lift you can ever prove. Below that floor, the honest moves are to aim at a bigger change, widen the market with new geographies or audiences, lengthen the window, or accept lower confidence and treat the read as directional.
Realistic baseline rates by channel
Your baseline rate is the single biggest input you can get wrong. Two cautions: an ad-click conversion rate (conversions ÷ clicks) runs far higher than a site-wide rate (conversions ÷ all visitors), and rates vary widely by industry. Use these public benchmarks as a sanity check, not a substitute for your own data.
| Context | Typical baseline | Use it as |
|---|---|---|
| Ecommerce site-wide CVR (global) | ~1.7–2.7% | Page CVR baseline |
| Google Ads conversion rate (all industries) | ~6.96% | Ad-click CVR baseline |
| Google Ads — auto repair / services | ~12.9% | High-intent ceiling |
| Google Ads — apparel / retail | ~3–4% | Lower-intent floor |
| Email click-through rate | ~2–3% | Creative CTR baseline |
What the experimentation field says
“95% significance is a very strong signal… aim for 95% or higher confidence. That is not always required, but it is recommended.”
“Any figure that looks interesting or different is usually wrong.” Twyman’s Law — the reason breakthrough results deserve the most scrutiny, not the least.
Peeking at a test repeatedly instead of once can more than double your true error rate — significance reached by stopping early is often an illusion.