The Bonferroni correction

The Bonferroni correction ensures the confidence level remains high when you're comparing more than two subgroups.

When comparing two groups, a 95% confidence level means that in situations where there are no actual differences in proportions between groups, we accept a false positives 5% of the time. However, false positives at a rate greater than 5% may be accidently accepted because we are making multiple comparisons. This is the case when comparing multiple groups and every group is getting compared using a confidence level of 95%.

For example, you're comparing different age groups response to a survey:

• Age 18-24
• Age 25-34
• Age 35-44
• Age 45-54

When you run the significance tests at 95% confidence, 6 tests are run to compare the different age groups:

• Test 1: 18-24 vs 25-34
• Test 2: 18-24 vs 35-44
• Test 3: 18-24 vs 45-54
• Test 4: 25-34 vs 35-44
• Test 5: 25-34 vs 45-54
• Test 6: 35-44 vs 45-54

If we looked at each comparison in these 6 tests individually, we would have a false positive 5% of the time where there is no difference in proportions. When all comparisons are made, the overall rate of false positives would be greater than 5% when there are no differences. If you apply the Bonferroni correction, it will ensure that the confidence level across all six tests remains at 95%. When applied the Bonferroni correction may lead to fewer significance differences being identified, but it will result in fewer false positives.