| Choosing an analysis to compare two groups
Go to the data or results table you wish to analyze (see "Entering data to compare two groups with a t test (or a nonparametric test)" on page 39). Press the Analyze button, and choose built-in analyses. Then select t tests from the Statistical Analyses section. If your table has more than two columns, select the two columns you wish to compare. (If you want to compare three or more columns, see "One-way ANOVA and nonparametric comparisons" on page 65. Press ok from the Analyze dialog to bring up the parameters dialog for t tests and related nonparametric tests:
Paired or unpaired test?
When choosing a test, you need to decide whether to use a paired test. Choose a paired test when the two columns of data are matched. Here are some examples:
- You measure a variable (perhaps, weight) before an intervention, and then measure it in the same subjects after the intervention.
- You recruit subjects as pairs, matched for variables such as age, ethnic group and disease severity. One of the pair gets one treatment, the other gets an alternative treatment.
- You run a laboratory experiment several times, each time with a control and treated preparation handled in parallel.
- You measure a variable in twins, or child/parent pairs.
More generally, you should select a paired test whenever you expect a value in one group to be closer to a particular value in the other group than to a randomly selected value in the other group.
Ideally, the decision about paired analyses should be made before the data are collected. Certainly the matching should not be based on the variable you are comparing. If you are comparing blood pressures in two groups, it is ok to match based on age or zip code, but it is not ok to match based on blood pressure.
t test or nonparametric test?
The t test, like many statistical tests, assumes that you have sampled data from populations that follow a Gaussian bell-shaped distribution. Biological data never follow a Gaussian distribution precisely, because a Gaussian distribution extends infinitely in both directions, and so it includes both infinitely low negative numbers and infinitely high positive numbers! But many kinds of biological data follow a bell-shaped distribution that is approximately Gaussian. Because ANOVA, t tests and other statistical tests work well even if the distribution is only approximately Gaussian (especially with large samples), these tests are used routinely in many fields of science.
An alternative approach does not assume that data follow a Gaussian distribution. In this approach, values are ranked from low to high and the analyses are based on the distribution of ranks. These tests, called nonparametric tests, are appealing because they make fewer assumptions about the distribution of the data. But there is a drawback. Nonparametric tests are less powerful than the parametric tests that assume Gaussian distributions. This means that P values tend to be higher, making it harder to detect real differences as being statistically significant. With large samples, the difference in power is minor. With small samples, nonparametric tests have little power to detect differences.
You may find it difficult to decide when to select nonparametric tests. You should definitely choose a nonparametric test in these situations:
- The outcome variable is a rank or score with only a few categories. Clearly the population is far from Gaussian in these cases.
- One, or a few, values are off scale, too high or too low to measure. Even if the population is Gaussian, it is impossible to analyze these data with a t test. Using a nonparametric test with these data is easy. Assign an arbitrary low value to values that are too low to measure, and an arbitrary high value to values too high to measure. Since the nonparametric tests only consider the relative ranks of the values, it won't matter that you didn't know one (or a few) of the values exactly.
- You are sure that the population is far from Gaussian. Before choosing a nonparametric test, consider transforming the data (i.e. to logarithms, reciprocals). Sometimes a simple transformation will convert non-Gaussian data to a Gaussian distribution.
In many situations, perhaps most, you will find it difficult to decide whether to select nonparametric tests. Remember that the Gaussian assumption is about the distribution of the overall population of values, not just the sample you have obtained in this particular experiment. Look at the scatter of data from previous experiments that measured the same variable. Also consider the source of the scatter. When variability is due to the sum of numerous independent sources, with no one source dominating, you expect a Gaussian distribution.
Prism performs normality testing in an attempt to determine whether data were sampled from a Gaussian distribution, but normality testing is less useful than you might hope. Normality testing doesn't help if you have fewer than a few dozen (or so) values.
Your decision to choose a parametric or nonparametric test matters the most when samples are small for reasons summarized here:
|
Large samples
(> 100 or so) |
Small samples
(<12 or so) |
|
Parametric tests
|
Robust. P value will be nearly correct even if population is fairly far from Gaussian.
|
Not robust. If the population is not Gaussian, the P value may be misleading.
|
| Nonparametric test |
Powerful. If the population is Gaussian, the P value will be nearly identical to the P value you would have obtained from a parametric test. With large sample sizes, nonparametric tests are almost as powerful as parametric tests.
|
Not powerful. If the population is Gaussian, the P value will be higher than the P value obtained from a t test. With very small samples, it may be impossible for the P value to ever be less than 0.05, no matter how the values differ.
|
|
Normality test
|
Useful. Use a normality test to determine whether the data are sampled from a Gaussian population.
|
Not very useful. Little power to discriminate between Gaussian and non-Gaussian populations. Small samples simply don't contain enough information to let you make inferences about the shape of the distribution in the entire population.
|
Assume equal variances?
The unpaired t test assumes that the two populations have the same variances. Since the variance equals the standard deviation squared, this means that the populations have the same standard deviation). A modification of the t test (developed by Welch) can be used when you are unwilling to make that assumption. Check the box for Welch's correction if you want this test.
This choice is only available for the unpaired t test. With Welch's t test, the degrees of freedom are calculated from a complicated equation and the number is not obviously related to sample size.
Welch's t test is used rarely. Don't select it without good reason.
One- or two-tail P value?
If you are comparing two groups, you need to decide whether you want Prism to calculate a one-tail or two-tail P value. To understand the difference, see One- vs. two-tail P values.
You should only choose a one-tail P value when:
- You predicted which group would have the larger mean (if the means are in fact different) before collecting any data.
- You will attribute a difference in the wrong direction (the other group ends up with the larger mean), to chance, no matter how large the difference.
Since those conditions are rarely met, two-tail P values are usually more appropriate.
Confirm test selection
Based on your choices, Prism will show you the name of the test you selected.
| Test |
Paired |
Nonparametric |
Unequal variances |
|
Unpaired t test
|
No
|
No |
No
|
| Welch's t test |
No
|
No |
Yes
|
| Paired t test |
Yes |
No |
N/A |
| Mann-Whitney |
No
|
Yes |
N/A
|
| Wilcoxon test |
Yes |
Yes |
N/A |
|
