Free tool
A/B test sample size calculator
This A/B test sample size calculator tells you how many visitors each landing page variant needs before a difference in lead conversion rate is worth trusting. Add your daily traffic and cost per click and it turns that into days and euros. It uses the standard two-proportion formula, at 95% significance and 80% power unless you change them.
The test
Traffic and cost
Visitors per variant
—
Total visitors
—
Estimated duration
—
Ad spend for the test
—
How to use the A/B test sample size calculator
It works out sample size and duration together. Here's what each input means and where to find it:
| Input | What it means | Where to get it |
|---|---|---|
| Baseline conversion rate | Share of visitors who submit the lead form on the current landing page. | Leads ÷ landing page visitors over the last two to four weeks, from the traffic you'll test on. |
| Minimum detectable effect (MDE) | The smallest lift you'd act on. A relative 20% on a 5% baseline means 5% → 6%; an absolute 1 pp means the same. | Your call: the lift that would justify switching landing pages. |
| Significance level | How strict the test is with false winners. At 95%, a variant that changes nothing still has a 5% chance of looking like a winner. | Keep 95% unless a wrong call would be expensive to undo. |
| Statistical power | The chance of catching a real lift as big as the MDE. At 80%, one real winner in five ends up looking like "no difference". | 80% is the convention. 90% costs about a third more visitors. |
| Variants | How many versions share the traffic, original included. | Two, unless traffic is plentiful. |
| Daily visitors | People who load the landing page each day, all variants combined. | Landing page views or sessions, not impressions or reach. |
| Cost per click | What one visitor costs you. Only the spend estimate uses it. | Cost per landing page view if you have it. Cost per link click underestimates the spend. |
The sample size formula
The calculator uses the standard normal-approximation formula for comparing two proportions, with unpooled variance and a two-sided test:
n = (z1−α/2 + z1−β)² × [p1(1 − p1) + p2(1 − p2)] ÷ (p2 − p1)²
- n is visitors per variant; p1 is the baseline rate, p2 the rate you want to detect.
- z1−α/2 and z1−β are standard normal quantiles: 1.95996 for 95% two-sided, 0.84162 for 80% power. The calculator computes them in your browser with Acklam's algorithm for the inverse normal distribution.
- With more than two variants, α is split across the comparisons with the original (Bonferroni correction): with three variants, each comparison runs at α = 0.025.
- Total visitors = n × number of variants. Duration = total ÷ daily visitors, rounded up to whole days.
Calculators that pool the variance under the null hypothesis get slightly larger samples: less than 1% more for baselines of 2% to 20% and lifts of 10% to 50%.
Worked example
Example: your landing page converts 5% of visitors into leads, you want to detect a 20% relative lift (5% → 6%), and you buy 300 visitors a day at €0.80 each.
- Variance term: 0.05 × 0.95 + 0.06 × 0.94 = 0.1039.
- Quantiles: (1.95996 + 0.84162)² = 7.84885.
- n = 7.84885 × 0.1039 ÷ 0.01² ≈ 8,154.96, rounded up to 8,155 visitors per variant.
- Total: 16,310 visitors. At 300 a day that's 55 days, or 8 full weeks.
- Spend: 16,310 × €0.80 = €13,048, to find out whether one landing page beats the other by a fifth.
For reference, here are the visitors per variant for two variants at 95% significance (two-sided) and 80% power. Columns are relative lifts:
| Baseline | +10% | +20% | +30% | +50% |
|---|---|---|---|---|
| 2% | 80,679 | 21,106 | 9,795 | 3,823 |
| 5% | 31,231 | 8,155 | 3,778 | 1,468 |
| 10% | 14,749 | 3,839 | 1,772 | 683 |
| 15% | 9,254 | 2,400 | 1,103 | 422 |
| 20% | 6,507 | 1,680 | 769 | 291 |
| 30% | 3,760 | 961 | 435 | 160 |
Halve the lift you want to detect and the sample roughly quadruples: a 10% lift on a 5% baseline needs 62,462 visitors in total.
How many visitors for an A/B test on a small account?
Usually more than the budget buys. The example above needs 8 weeks of traffic and €13,048; at 100 visitors a day it would take 164 days. And for the whole test, half the visitors go to whichever landing page is worse. If B really is 20% better, the 50/50 split sends 8,155 visitors to A and costs you about 82 leads (8,155 × 1 pp) you'd have got by sending them to B.
You have three ways out:
- Test bigger changes. Doubling the MDE to 40% cuts the example to 2,210 visitors per variant: 15 days instead of 55. A new offer or a shorter form can move conversion that much; a button colour can't. The landing page A/B testing guide ranks what's worth testing.
- Run fewer variants. Each extra variant needs its own sample and tightens the Bonferroni correction. Three variants push the example to 29,628 visitors, 99 days.
- Let a bandit allocate the traffic. Thompson Sampling moves visitors toward the landing page that is ahead as evidence arrives, so a clear loser stops eating budget early. You give up two things a fixed test promises: a known false-positive rate and a planned end date. In exchange you can read, at any moment, the probability that each variant is the best.
DropCPL's automatic A/B testing works this way: landing page variants for the same creative, with Thompson Sampling deciding which one each visitor sees.
When the formula asks for more traffic than you buy, let DropCPL move it to the winner as evidence comes in. Its automatic A/B testing allocates visitors between landing page variants with Thompson Sampling, and every lead stays tied to the creative and the landing page that produced it. Start free →
The peeking problem
The sample size only protects you if you read the result once, when the test reaches it. Check every morning and stop the first time p dips below 0.05, and the 5% false-positive rate quietly becomes much higher. Evan Miller's classic analysis puts numbers on it: peek ten times and a result that looks significant at 1% is really significant only at 5%. In his worst case, a significance test after every single observation, a change that does nothing comes out significant 26.1% of the time.
Fix the sample before the test starts. Run it for at least seven full days, since Monday and Sunday traffic are different populations and you want both. Then read the outcome once with the A/B test significance calculator. If you need to look while it runs, use a method designed for it: a sequential test, or a bandit, which doesn't test a hypothesis in the first place.
Common mistakes with A/B test sample size
- Using an account-wide conversion rate. The baseline has to come from the landing page and the traffic you are testing. A 12% account average says nothing about a cold-audience landing page converting at 4%.
- Setting the MDE to the lift you hope for. Hoping for +50% gives a small, comfortable sample; then a real +15% lift goes undetected.
- Counting link clicks as visitors. Not every click loads the landing page, so daily traffic counted in clicks overstates visitors and understates the duration. More in landing page views vs link clicks.
- Stopping when it looks good, or extending until it does. Both break the error rates the sample size was built on.
- Changing the traffic mid-test. New creatives or audiences halfway through change who reaches the landing page. Keep the creative fixed, or read results per creative and landing page pair.
FAQ
How many visitors do I need for an A/B test?+
It depends on your baseline conversion rate and the smallest lift you want to detect. At a 5% baseline, detecting a 20% relative lift (5% to 6%) at 95% significance and 80% power takes 8,155 visitors per variant, 16,310 in total. Detecting a 10% lift at the same baseline takes 31,231 per variant.
How long should an A/B test run?+
Until every variant reaches the sample size you computed before starting, and never less than seven full days, so each weekday is in the sample. Round up to whole weeks when you can. Stopping early because the numbers look good inflates the false-positive rate.
What minimum detectable effect should I use for a landing page test?+
The smallest lift that would make you switch landing pages, not the lift you hope to see. If the calculator says your traffic can't detect that lift in a reasonable time, test a bolder change, such as the offer or the length of the form, or use bandit allocation instead of a fixed split.
Why do other sample size calculators give a different number?+
They make different choices: pooled or unpooled variance, one-sided or two-sided tests, continuity corrections. This calculator uses the unpooled two-sided formula; the pooled version gives 8,158 instead of 8,155 in the 5% to 6% example. A one-sided test needs about a fifth fewer visitors, because it only looks for a lift in one direction.
Can I stop an A/B test as soon as it's significant?+
Not with a fixed-sample test. The 95% guarantee assumes you look once, at the planned sample size; checking repeatedly and stopping at the first significant reading produces winners that are noise. If you need to decide while the test runs, use a sequential method or a bandit such as Thompson Sampling.
How does the calculator handle more than two variants?+
It compares each challenger with the original and divides the significance level by the number of comparisons (Bonferroni correction). With three variants at 95%, each comparison runs at 97.5%, which raises the sample from 8,155 to 9,876 visitors per variant in the 5% to 6% example.