P value
The Kruskal-Wallis test is a nonparametric test that compares three or more unmatched groups. To perform this test, Prism first ranks all the values from low to high, paying no attention to which group each value belongs. The smallest number gets a rank of 1. The largest number gets a rank of N, where N is the total number of values in all the groups. The discrepancies among the rank sums are combined to create a single value called the Kruskal-Wallis statistic (some books refer to this value as H). A large Kruskal-Wallis statistic corresponds to a large discrepancy among rank sums.
The P value answers this question:
If your samples are small (even if there are ties), Prism calculates an exact P value. If your samples are large, it approximates the P value from a Gaussian approximation (based on the fact that the Kruskal-Wallis statistic H approximates a chi-square distribution. Prism labels the P value accordingly as exact or approximate. Here, the term Gaussian has to do with the distribution of sum of ranks and does not imply that your data need to follow a Gaussian distribution. The approximation is quite accurate with large samples and is standard (used by all statistics programs). The exact calculations can be slow with large(ish) data sets or slow(ish) computers. You can cancel the calculations in that case, by clicking the cancel button on the progress dialog. If you cancel computation of the exact P value, Prism will instead show the approximate P value.
If the P value is small, you can reject the idea that the difference is due to random sampling, and you can conclude instead that the populations have different distributions.
If the P value is large, the data do not give you any reason to conclude that the distributions differ. This is not the same as saying that the distributions are the same. Kruskal-Wallis test has little power. In fact, if the total sample size is seven or less, the Kruskal-Wallis test will always give a P value greater than 0.05 no matter how much the groups differ.
Tied values
The Kruskal-Wallis test was developed for data that are measured on a continuous scale. Thus you expect every value you measure to be unique. But occasionally two or more values are the same. When the Kruskal-Wallis calculations convert the values to ranks, these values tie for the same rank, so they both are assigned the average of the two (or more) ranks for which they tie.
Prism uses a standard method to correct for ties when it computes the Kruskal-Wallis statistic.
There is no completely standard method to get a P value from these statistics when there are ties. Prism 6 and later handles ties differently than did prior versions. Prism will compute an exact P value with moderate sample sizes. Earlier versions always computed an approximate P value when there were ties. Therefore, in the presence of ties, Prism 6 and later may report a P value different than that reported by earlier versions of Prism or by other programs.
If your samples are small, Prism calculates an exact P value. If your samples are large, it approximates the P value from the chi-square distribution. The approximation is quite accurate with large samples. With medium size samples, Prism can take a long time to calculate the exact P value. While it does the calculations, Prism displays a progress dialog and you can press Cancel to interrupt the calculations if an approximate P value is good enough for your purposes. Prism always reports whether the P value was computed exactly or via an approximation.
Dunn's test
Dunn's multiple comparisons test compares the difference in the sum of ranks between two columns with the expected average difference (based on the number of groups and their size).
For each pair of columns, Prism reports the P value as >0.05, <0.05, <0.01, or <0.001. The calculation of the P value takes into account the number of comparisons you are making. If the null hypothesis is true (all data are sampled from populations with identical distributions, so all differences between groups are due to random sampling), then there is a 5% chance that at least one of the post tests will have P<0.05. The 5% chance does not apply to each comparison but rather to the entire family of comparisons.
For more information on the post test, see Applied Nonparametric Statistics by WW Daniel, published by PWS-Kent publishing company in 1990 or Nonparametric Statistics for Behavioral Sciences by S. Siegel and N. J. Castellan, 1988. The original reference is O.J. Dunn, Technometrics, 5:241-252, 1964.
Prism refers to the post test as the Dunn's post test. Some books and programs simply refer to this test as the post test following a Kruskal-Wallis test, and don't give it an exact name.
Analysis checklist
Before interpreting the results, review the analysis checklist.
