Bonferroni and Sidak tests in Prism

Prism can perform Bonferroni and Sidak multiple comparisons tests as part of several analyses:

•Following one-way ANOVA. This makes sense when you are comparing selected pairs of means, with the selection based on experimental design. Prism also lets you choose Bonferroni tests when comparing every mean with every other mean. We don't recommend this. Instead, choose the Tukey test if you want to compute confidence intervals for every comparison or the Holm-Šídák test if you don't.

•Following two-way ANOVA. If you have three or more columns, and wish to compare means within each row (or three or more rows, and wish to compare means within each column), the situation is much like one-way ANOVA. The Bonferroni test is offered because it is easy to understand, but we don't recommend it. If you enter data into two columns, and wish to compare the two values at each row, then we recommend the Bonferroni method, because it can compute confidence intervals for each comparison. The alternative is the Holm-Šídák method, which has more power, but doesn't compute confidence intervals.

•As part of the analysis that performs many t tests at once.

•To analyze a stack of P values.

Key facts about the Bonferroni and Šídák methods

•The inputs to the Bonferroni and Šídák (the letter Š is pronounced "Sh") methods are a list of P values, so these methods can be used whenever you are doing multiple comparisons. They are not limited to use as followup tests to ANOVA.

•It only makes sense to use these methods in situations for which a specialized test has not been developed. For example, use the Tukey method when comparing every mean with every other mean, and use Dunnett's method to compare every mean with a control mean. But use Bonferroni or Šídák when you select a set of means to compare.

•The Bonferroni and Šídák methods can determine statistical significance, compute adjusted P value, and also compute confidence intervals.

•The Šídák method has a bit more power than the Bonferroni method.

•The Šídák method assumes that each comparison is independent of the others. If this assumption is independence cannot be supported, choose the Bonferroni method, which does not assume independence.    

•The Bonferroni method is used more frequently, because it is easier to calculate (which doesn't matter when a computer does the work), easier to understand, and much easier to remember.

•Prism 5 and earlier offered  the Bonferroni method, but not the Šídák method.

•The Bonferroni method is sometimes called the Bonferroni-Dunn method. And the Šídák method is sometimes called the Bonferroni-Šídák method.

References

1. H Abdi. The Bonferonni and Šidák Corrections for Multiple Comparisons. In N.J. Salkind (Ed.), 2007, Encyclopedia of Measurement and Statistics. Thousand Oaks (CA): Sage. pp. 103-107.

2. DJ Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, Fifth edition, 2011, ISBN=978-7-1398-5801-1