Free research tool
Significance Calculator
Check whether the gap between two groups is real or sampling noise. Compare two percentages (crosstabs, brand awareness by city, A/B conversion) or two averages (ratings, spend), and get the p-value, confidence interval and the sample you’d need.
Free, no sign-up. Last reviewed against source documents on .

Use for anything counted as yes/no: % who prefer a pack, top-2-box, conversion in an A/B test, % aware of a brand in two cities.
ResultSignificantDetails ↓How big a gap is significant?
The smallest difference between two equal-size groups that is significant, for results near 50% (the hardest case).
| People per group | 90% confidence | 95% confidence | 99% confidence |
|---|---|---|---|
| 100 | 11.6 pts | 13.9 pts | 18.2 pts |
| 200 | 8.2 pts | 9.8 pts | 12.9 pts |
| 300 | 6.7 pts | 8.0 pts | 10.5 pts |
| 500 | 5.2 pts | 6.2 pts | 8.1 pts |
| 750 | 4.2 pts | 5.1 pts | 6.7 pts |
| 1,000 | 3.7 pts | 4.4 pts | 5.8 pts |
| 2,000 | 2.6 pts | 3.1 pts | 4.1 pts |
| 5,000 | 1.6 pts | 2.0 pts | 2.6 pts |
For results near 10% or 90% the required gap is smaller (about 60% of these values).
How the tests work
Two percentages:
Two averages (Welch):
Worked example: 22% of 1,000 people in Mumbai know a brand vs 18% of 1,000 in Delhi. Pooled p̄ = 0.20, z = 0.04 ÷ 0.0179 = 2.24, p = 0.025: significant at 95%, not at 99%.
Planning the study? The sample size calculator sizes each group. Comparing loyalty scores? The NPS calculator has its own test.
Questions, answered
What does statistically significant mean in a survey?
It means the difference between two groups is larger than you would expect from sampling noise alone. At 95% confidence, a significant result has less than a 5% chance of appearing if the two groups were truly the same (p < 0.05). It does not mean the difference is large or important.
Which test does this calculator use?
For percentages it uses a two-proportion z-test with a pooled standard error, and an unpooled interval for the gap. For averages it uses Welch’s t-test, which does not assume the two groups have equal variance, so it is safe for unequal group sizes.
How big a difference is significant with 500 people per group?
Around 6.2 pts at 95% confidence when results are near 50%. With 200 per group you need about 9.8 pts; with 1,000 per group about 4.4 pts.
Can I use this for an A/B test?
Yes. Enter visitors as the sample size and conversions as the count for each variant. Decide the sample size before the test and avoid stopping the moment it turns significant, since repeated peeking inflates false positives.
Can I compare a subgroup with the total?
Not directly with this test, because the subgroup is part of the total and the samples are not independent. Compare the subgroup with everyone else instead (for example, women vs men, not women vs all).
What is the p-value?
The probability of seeing a difference at least this large if there were truly no difference. A p-value of 0.03 is significant at 95% confidence (below 0.05) but not at 99% (above 0.01).
Sources and method
Two-proportion z-test (pooled SE for the test, unpooled for the interval), Welch’s t-test with Welch–Satterthwaite degrees of freedom, and the standard two-proportion power formula at 80% power. p-values are checked against SciPy in automated tests.
More free research tools

Sample Size Calculator
Completes you need, and how many people to reach after incidence

Margin of Error Calculator
How precise is your survey result? Get the ± and the range

NCCS Calculator
Classify any Indian household into NCCS A1 to E3 in two questions

ISEC Calculator
MRSI’s new socio-economic classes, ISEC 1 to 12, for any household
Found a real difference? Dig into why.
Run a follow-up survey in PollPe with open-ended questions and reach the segment that moved.