How can a multiple comparisons test after ANOVA be labeled with an asterisk, but not be statistically significant?
At first glance, these results shown below from multiple comparisons after one-way ANOVA seem contradictory. For both the first and third comparison, the summary is * but the answer to the question "Significant?" is No
The reason for this apparent discrepancy is simple. Alpha was set to 0.01 for the mutliple comparisons testing. This is done in Prism 6 in the Options tab of the ANOVA dialog.
When Prism answers the question "Significant?", it uses the alpha value chosen in the dialog. For the first and third comparisons shown above the multiplicity adjusted P value is greater than 0.01 (the value chosen for alpha), so the answer is "No".
In contrast, the P value summary always uses the same scale to decide how many asterisks (if any) to assign and pays no attention to the value of alpha. P values between 0.01 and 0.05 are assigned one asterisks, regardless of the value of alpha.
Keywords: multiple comparisons,