Two-proportion z-test calculator

What does a two-proportion z-test compare?

Whether two groups have the same underlying proportion — for example, the share of patients helped by two medications. The null hypothesis is that the two population proportions are equal, so the difference between them is 0.

How is z calculated?

z = (p₁ − p₂) ÷ √[p̂(1 − p̂)(1/n₁ + 1/n₂)], where p̂ = (x₁ + x₂) ÷ (n₁ + n₂) is the pooled proportion (OpenStax Introductory Statistics). Because the null hypothesis says both groups share one proportion, the best estimate of it uses all the data, which is why the two samples are pooled in the standard error. The z-score is then compared with the standard normal distribution to get the p-value.

Example

OpenStax compares two hives medications, with 20 of 200 patients in one group and 12 of 200 in the other. The sample proportions are 0.10 and 0.06, the pooled proportion is 0.08, and the standard error is √(0.08 × 0.92 × (1/200 + 1/200)) = 0.0271. So z = 0.04 ÷ 0.0271 = 1.47 and the two-sided p-value is 0.1404 — no significant difference at α = 0.01.

Worked example with a one-sided test: seat belts

OpenStax (Example 10.9) asks whether a smaller share of women than men do not wear seat belts. In the sample, 156 of 2,169 women and 183 of 2,231 men did not: 7.19% against 8.20%. The pooled proportion is 0.077, z = −1.256, and because the claim is that women’s share is lower, the p-value is the left tail, 0.1045. The two-sided p-value would be 0.2089, twice as large. At α = 0.01 neither is significant, so there is not enough evidence that women’s share is lower.

One-sided or two-sided?

Use two-sided unless, before seeing the data, you only cared about a difference in one direction. The calculator shows both.

When is the test reliable?

When each group has at least about 5 successes and 5 failures and the samples are independent random samples. OpenStax also asks that each population be at least 10 times larger than its sample. With fewer events, use Fisher’s exact test.

Common mistakes with the two-proportion test

First, entering percentages: the fields take the number of successes and the sample size, as whole numbers. Second, treating “not significant” as “no difference”. In the hives example the 4-point gap is real in the sample, but with 200 patients per group the power to detect a true gap of 0.10 against 0.06 is only 31%; about 721 per group would give 80%. Third, applying the test to the same people measured twice, which needs a paired method. Fourth, comparing groups that overlap, such as “all customers” against “customers who bought”. Each person must belong to only one sample.

How does this relate to the chi-square test?

For a 2 × 2 table the two tests are the same: z² equals the chi-square statistic of independence. For the hives data z² = 2.17 and the chi-square statistic of the table [20, 180; 12, 188] is 2.17. The z-test has the advantage of allowing a one-sided alternative.

Test or confidence interval?

The test answers a yes-or-no question about whether the proportions differ; a confidence interval for the difference shows how big the difference might be. The two use different standard errors. The test pools the samples because it assumes the proportions are equal, while the interval does not, using √(p₁(1 − p₁) ÷ n₁ + p₂(1 − p₂) ÷ n₂). For the hives data that standard error is 0.02706 against the pooled 0.02713 (close here because the two samples are the same size and the proportions are near each other), and the 95% interval for the difference is −0.013 to 0.093. It contains 0, which agrees with a test that is not significant at 5%, and it also shows that the true gap could be anywhere from slightly negative to about 9 points. When the test and the interval seem to disagree near the boundary, the difference in standard errors is the reason; the interval is the better one to report.

Checking the conditions on real numbers

For the hives data the successes are 20 and 12 and the failures are 180 and 188, all well above 5, and each sample of 200 is a small share of its population. If one group had only 3 successes, or the same 200 people appeared in both groups, the normal approximation would not hold, and Fisher's exact test or a paired method would be the right tool. Also note what the test cannot do: a significant result shows that the proportions differ in the populations sampled, not why they differ, unless the groups were formed by random assignment.

Sources: OpenStax, Introductory Statistics 2e, section 10.3 (Comparing Two Independent Population Proportions, Examples 10.8 and 10.9). Related: chi-square test of independence, Fisher’s exact test calculator, statistical power calculator, relative risk calculator and sample size calculator.