Why we recommend you do not use the Newman-Keuls multiple comparison test

The Newman-Keuls (also called Student-Newman-Keuls test)  compares all pairs of means following one-way ANOVA.

The Newman-Keuls test has more power than the Tukey test, which also compares all means. It can sometimes find that a difference between two groups is 'statistically significant' in some cases where the Tukey test would conclude that the difference is 'not statistically significant'. But this extra power comes at a price. We suggest that you avoid the Newman-Keuls test for three reasons:

• Although the whole point of multiple comparison post tests is to keep the chance of a Type I error in any comparison to be 5%, in fact the Newman-Keuls test doesn't do this¹. In some cases, the chance of a Type I error can be greater than 5%. (The Newman-Keuls test works fine with three groups; the increase in Type I error occurs only with four or more groups.)
• Because the Newman-Keuls test works in a sequential fashion, it can not produce 95% confidence intervals for each difference or multiplicity adjusted exact P values.  In contrast, the Tukey test can compute both confidence intervals and adjusted P values.
• It is difficult to articulate exactly what null hypotheses the Newman-Keuls test actually tests, so difficult to interpret its results.

 If you want to obtain confidence intervals as well as statements of signficance, use Tukey's test. If you don't need confidence intervals, you can get a bit more power by using the Holm-Sidak test.

¹ MA Seaman, JR Levin and RC Serlin, Psychological Bulletin 110:577-586, 1991.