Choosing when to use a nonparametric test is not straightforward. Here are some considerations:
•Off-scale values. With some kinds of experiments, one, or a few, values may be "off scale" -- too high or too low to measure. Even if the population is Gaussian, it is impossible to analyze these data with a t test or ANOVA. If you exclude these off scale values entirely, you will bias the results. If you estimate the value, the results of the t test depend heavily on your estimate. The solution is to use a nonparametric test. Assign an arbitrary low value to values that are too low to measure, and an arbitrary high value to values too high to measure. Since the nonparametric tests only analyze ranks, it will not matter that you don't know one (or a few) of the values exactly, so long as the numbers you entered gave those values the correct rank.
•Transforming can turn a nongaussian distribution into a Gaussian distribution. If you are sure the data do not follow a Gaussian distribution, pause before choosing a nonparametric test. Instead, consider transforming the data, perhaps using logarithms or reciprocals. Often a simple transformation will convert non-Gaussian data to a Gaussian distribution. Then analyze the transformed values with a conventional test.
•Noncontinuous data. The outcome is a rank or score with only a few categories. Clearly the population is far from Gaussian in these cases. The problem with using nonparametric tests is that so many values will tie for the same rank. Nonparametric tests have special corrections built-in to deal with tied ranks, but I am not sure how well those work when there are lots of tied ranks. An alternative would be to do a chi-square test.
•Small samples. If you have tiny samples (a few subjects in each group), the nonparametric tests have little or no power to find a significant difference.
•Normality tests should not be used to automatically decide whether or not to use a nonparametric test. But they can help you make the decision.
•You really should choose your statistical test as part of the experimental design. If you try this test, then that test, until you get a result you like, you are likely to be mislead.
