Power Analysis Calculator
How many people per group do you need to detect a real difference? Set the effect you care about, α and power; get n per group, or the power you already have.
Why power analysis comes before the study
A study with too few participants can miss a real effect and report “no significant difference”, which is easy to misread as “no difference”. Power analysis fixes the sample size in advance from four inputs: the effect you want to detect, the significance level α, the power you want, and whether the test is one- or two-tailed. Change any one and the others move: smaller effects, stricter α and higher power all need more people.
When the study is done, analyse it with the matching test: the t-test calculator for means or the chi-square test for two proportions. To size a single survey estimate instead, use the sample size calculator. The effect-size benchmarks come from Jacob Cohen’s Statistical Power Analysis for the Behavioral Sciences (1988).
Frequently asked questions
What is statistical power?
Power is the probability that a study detects an effect of a given size when that effect really exists, that is, the chance of correctly rejecting a false null hypothesis. It equals 1 − β, where β is the Type II error rate. Most studies aim for 80% or 90%.
How do I calculate sample size for a t-test?
For two independent groups with equal size: n per group ≈ 2 (z₁₋α/₂ + z₁₋β)² ÷ d², where d is Cohen’s d (difference ÷ SD). At α = 0.05 two-tailed and 80% power, d = 0.5 needs about 63 per group by the normal formula, and 64 once the t-distribution is used. This calculator iterates with t critical values, which matches standard software closely.
What effect size should I use?
Ideally the smallest difference that would matter in practice, or an estimate from a pilot study or earlier research. Cohen’s rough benchmarks are d = 0.2 (small), 0.5 (medium) and 0.8 (large), but he intended them only as a fallback when nothing better is known.
How do I size an A/B test?
Choose “Two proportions”, enter the current conversion rate as group 1 and the smallest improved rate worth detecting as group 2. For 10% vs 12% at α = 0.05 and 80% power you need about 3,840 visitors per variant. Small lifts on low baseline rates need large samples.
Should I use a one-tailed or two-tailed test?
Use two-tailed unless you decided before collecting data that only one direction matters and an effect in the other direction would be treated as no effect. One-tailed tests need fewer participants, which is exactly why reviewers are wary of them.
What about dropouts?
The result is the number of participants you need at analysis. If you expect 15% to drop out, divide by 0.85 and recruit that many.