Statistical power calculator
How many people do you need? Or how likely is your test to find a real effect?
What is statistical power?
Power is the probability that a test rejects the null hypothesis when a real effect of the size you assumed exists: power = 1 − β, where β is the chance of missing it. It depends on four things — the effect size, the sample size, the significance level α and whether the test is one- or two-sided. Fix any three and the fourth follows; a power of 80% is the usual target.
How big a sample do I need? Example
To detect a medium effect (d = 0.5) with a two-sided two-sample t-test at α = 0.05 and 80% power you need 64 people per group, 128 in total; the power with exactly 64 is 80.15%. The exact answer is 63.77, which is rounded up. For 90% power it is 86 per group, and for 95% it is 105. A one-sample or paired t-test with d = 0.5 needs 34.
What is Cohen’s d, and what are small, medium and large?
Cohen’s d is the difference between means divided by the standard deviation. Cohen (1988) suggested d = 0.2, 0.5 and 0.8 as conventional small, medium and large effects when nothing better is known. They are rough benchmarks, not thresholds: use an effect that matters in your field. To turn your own data into d, use the effect size calculator.
Which tests does this calculator cover?
The two-sample t-test (equal group sizes and a common standard deviation), the one-sample t-test, the paired t-test (enter the mean and standard deviation of the differences) and the test for two proportions. The t-test results use the noncentral t distribution, not a normal approximation, and agree with R’s power.t.test: for n = 20 per group and d = 1 the power is 86.90%, and 90% power needs 22.02 per group, rounded up to 23. The proportion test uses the normal approximation of R’s power.prop.test.
Should I choose a one-sided or a two-sided test?
A one-sided test puts all of α in one tail, so it has more power for an effect in that direction and none for the other. With 20 per group and d = 1, power is 92.8% one-sided against 86.9% two-sided. Choose the direction before you see the data, and use two-sided unless a difference in the other direction would truly be meaningless.
How do I find the sample size for two proportions?
Enter the two proportions you expect, for example 0.50 and 0.75. With 50 per group the power is 74.0%, and 90% power needs 77 per group (76.7 before rounding up) — the examples in R’s power.prop.test help. The formula is an approximation, so very small samples or proportions near 0 or 1 call for an exact test.
How is this different from the sample size calculator?
The sample size calculator finds how many survey responses give a chosen margin of error. This one finds how many observations a hypothesis test needs to detect a given effect with a given power.
Sources: R Core Team, documentation of power.t.test and power.prop.test (stats package); J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. (1988), for the d = 0.2 / 0.5 / 0.8 conventions; R. V. Lenth (1989), Algorithm AS 243, for the noncentral t distribution. Related: effect size calculator, sample size calculator and t-test calculator.