Starting with Prism 8, one-way ANOVA allows you to specify that you don't wish to assume that all the groups are sampled from populations with the same SD (and thus the same variances). When you make this choice, Prism performs both Brown-Forsythe ANOVA and Welch ANOVA. When you make this choice, Prism offers a different set of multiple comparisons tests.
Hypothesis testing approach
If you choose the statistical hypothesis testing approach, Prism offers three tests: Dunnett T3, Games and Howell, and Tamhane T2. We recommend Dunnett T3 when sample sizes are small (<50 per group) and Games and Howell when samples are larger.
A multiple comparisons procedure starts by calculating the ratio of the difference between a pair of means divided by the standard error of that difference.
When you assume that all variances are equal (as is usual with ANOVA), the multiple comparisons procedures pool the SDs from all the groups, and uses the combined sample size of all groups when computing the degrees of freedom. This computed from the Mean Square Error (or Residual) in the ANOVA table. This gives a more precise measure of population scatter, which gives the procedures a bit more power.
If you don't assume that all variances are equal, then when comparing two means the multiple comparison procedure uses only the SD and sample sizes of those two groups. The ratio is the t ratio, just as if you were only comparing those two groups. The degrees of freedom are computed from a complicated equation that accounts for unequal sample size and unequal SDs. The P values are computed from t and df, accounting for multiple comparisons.
The three tests compute the same t ratio and the same df values. They differ in how they compute the P value from t and df.
The details are explained in a good review by Shingala(1). However, Shingala mistakenly says that the Dunnett's T3 test is designed (like the test Dunnett is best know for) for comparing each mean to a control mean. Not so. It is designed to compare all pairs of means(2).Tamhane (3) has recommended some modifications of these tests in some situations, but these modfifcations are not standard so Prism uses the original versions.
We recommend Dunnett T3 when sample sizes are small (<50 per group) and Games and Howell when samples are larger.
FDR approach
If you use one of the methods based on controlling the FDR, Prism first computes individual P values using the Welch-correct t test, and uses those P values in the FDR method. It does not pool the standard deviations as is done with the Fisher LSD test, because it only makes sense to pool variances when you assume the population variances are equal.
Don't correct for multiple comparisons
If you choose to not correct for multiple comparisons, Prism computes the P values using a t test with the Welch correction. It does not pool the standard deviations as is done with the Fisher LSD test, because it only makes sense to pool variances when you assume the population variances are equal.
References
1.Shingala, M.C., and Rajyaguru, A. (2015). Comparison of post hoc tests for unequal variance. International Journal of New Technologies in Science and Engineering
2.Dunnett, C.W. (1980). Pairwise Multiple Comparisons in the Unequal Variance Case. Journal of the American Statistical Association 75: 796–800.
3.Tamhane, A.C. (1979). A comparison of procedures for multiple comparisons of means with unequal variances. Journal of the American Statistical Association 74: 471–480.
