Skip to the calculator
Sample size31,234 per variant 63 daysEdit inputs
DocsFree trial

A/B test sample size calculator

How many visitors each variant needs, and how long that takes at your traffic. It all runs in your browser.

%
Your conversion rate today
%
Relative lift, so 10% takes 5% to 5.5%
Entering the test, all variants together
Control counts as one
Significance and power 95% confidence, 80% power

Example result

Visitors per variant
31,234
Days to run
63

62,468 visitors in total, about 9 weeks at 1,000 a day. That's enough to detect a change from 5% to 5.5%.

What the wait looks like

Example inputs 5% baseline, a 10% relative lift, 1,000 visitors a day, 2 variants.

Day63of 63

31,234 of 31,234 visitors per variant

Sample reached on day 63. Read the result now.

Each square is a day

How it's worked out

A standard two-proportion power calculation, the one a z-test between two conversion rates is built on.

n =( z1−α/2·√2p̄(1−p̄) + z1−β·√p₁(1−p₁) + p₂(1−p₂) )²(p₂ − p₁)²

With these inputs p₁ = 0.05, p₂ = 0.055, z1−α/2 = 1.96, z1−β = 0.842, so n = 31,234 per variant.

p₁ and p₂
p₁ is your baseline rate. p₂ is the rate you want to be able to spot: p₁ × (1 + the effect). p̄ is the average of the two.
z1−α/2
How far into the tail of a normal curve a result has to land to count as significant. At alpha 0.05, two-sided, it's 1.96.
z1−β
Sets the power: the chance the test spots the effect if it's really there. At 80% power it's 0.84.
n
Visitors per variant, rounded up. Times the number of variants gives the total. The total divided by visitors a day, rounded up, gives the days.
More than two variants
Each variant is compared with control, so alpha is split evenly across those comparisons (a Bonferroni correction). That's why each variant needs more visitors when you add one.

Before you start

  • Fix the length up front and read the result once, at the end. Checking every day and stopping at the first significant result makes false positives far more likely than alpha says.
  • Run whole weeks if you can, so weekdays and weekends both count.
  • Pick the smallest lift worth knowing about, not the one you hope for. Halving the effect needs about four times the visitors.

Embed this calculator

Put the sample size calculator on your own site or docs. It runs in the visitor's browser, sets no cookies and sends nothing anywhere. Keep the credit link under it, please.

At 1,000 visitors a day, this test needs 63 days.

Not enough traffic for a fixed test? A bandit moves traffic to the leader as it learns. How bandits work in evolveAB

Start a 30-day free trialNo card needed.

When the test is done, read it with the significance calculator.