Two-sample t-test calculator

What does a two-sample t-test tell me?

Whether the difference between two group means is larger than you would expect by chance. Group 1 with mean 20 (SD 4, n 30) against group 2 with mean 18 (SD 5, n 35) gives t = 1.79 and p = 0.078 by Welch's test.

What is Welch's t-test?

A version that does not assume the two groups have equal variances. It adjusts the degrees of freedom (62.7 in the example) and is the safer default.

When should I use the pooled (Student) test?

Only when you have good reason to think the two groups have the same variance. It uses n₁ + n₂ − 2 degrees of freedom — 63 in the example, where it gives t = 1.76 and p = 0.083.

Can I use it for before-and-after data on the same people?

No — that needs a paired t-test on the differences. This calculator is for two independent groups.

What does the confidence interval of the difference show?

The range of plausible values for the true difference in means. If it includes 0, the difference is not significant at that level.

Does a p-value of 0.078 mean the groups are the same?

No. A p-value above 0.05 means the data do not provide strong enough evidence of a difference at that level; it does not show that the means are equal. In the example the observed difference is 2.0 points, and the 95% confidence interval runs from -0.23 to 4.23, which includes 0 but also includes differences of several points. A larger sample might narrow it.

Is this a one-tailed or two-tailed test?

Two-tailed: the p-value counts differences in either direction. If you decided before collecting data that only one direction matters, the one-tailed p-value is half of the two-tailed one when the difference is in that direction. Choosing the direction after you see the result is not valid.

What is computed

The t statistic is the difference between the two sample means divided by its standard error: t = (x̄₁ − x̄₂) ÷ SE. For Welch's test, SE = √(s₁²/n₁ + s₂²/n₂), and the degrees of freedom come from the Welch-Satterthwaite formula, which is why they are often not whole numbers. The pooled test replaces the two variances with one weighted average and uses n₁ + n₂ − 2 degrees of freedom. The p-value is the probability, if the true means were equal, of a t at least as far from zero as the one observed, using Student's t distribution. The test assumes independent, random samples and means that are roughly normal, which is reasonable for moderate or large samples even when the raw data are skewed.

Worked example: two groups of 30 and 35

Group 1: mean 20, SD 4, n = 30. Group 2: mean 18, SD 5, n = 35. The difference is 2.0. SE = √(16/30 + 25/35) = 1.1170. So t = 2.0 ÷ 1.1170 = 1.791 with 62.7 degrees of freedom, and p = 0.0782. The 95% critical value for those degrees of freedom is 1.999, so the interval for the difference is 2.0 ± 1.999 × 1.1170, which is -0.23 to 4.23. Because 0 is inside, the result is not significant at the 0.05 level. A stronger case: means of 105 and 98 with SDs 15 and 14 and sample sizes 40 and 38 give t = 2.132 and p = 0.0363.

Significant versus important

A p-value says how surprising the data are under "no difference"; it does not say how large or useful the difference is. With big samples tiny differences become significant: a half-point gap with 3,000 and 3,500 people per group gives p = 0.00001. An effect size helps: Cohen's d is the difference divided by the pooled standard deviation, 2.0 ÷ 4.57 = 0.44 in the example. Report the difference and its confidence interval, not only whether p crossed 0.05.

Common mistakes

Using this test for before-and-after measurements on the same people is the main one; those pairs are not independent and need a paired t-test on the differences. Entering standard errors where standard deviations are expected makes t much too large. Running many tests and reporting only the significant one inflates the false-positive rate. Using the pooled version when the groups clearly differ in spread, especially with unequal sample sizes, can distort the p-value. And with very small samples and strongly non-normal data, the t distribution may not be a good approximation at all.

Sources: OpenStax Introductory Statistics 2e, chapter 10 (hypothesis testing with two samples: comparing two independent population means, sections 10.1 and 10.2); NIST/SEMATECH e-Handbook of Statistical Methods, two-sample t-test for equal means (Welch's approximation). Related: confidence interval calculator, p-value calculator and standard error calculator.