Frequently Asked Questions
What is difference between Type I, Type II, and Type III errors? And what is a Type 0 error?
FAQ# 1080 Last Modified 1-January-2009
A Type I error occurs when there really is no difference (association, correlation..) overall, but random sampling caused your data to show a statistically significant difference (association, correlation...). So your conclusion that the two groups are really different (associated, correlated) is an error.
A Type II error occurs when there really is a difference (association, correlation) overall, but random sampling caused your data to not show a statistically significant difference. So your conclusion that the two groups are not really different is an error.
The term Type III error has two different meanings.
One definition (attributed to Howard Raiffa) is that a Type III error occurs when you get the right answer to the wrong question. This is sometimes called a Type 0 error.
Another definition is that a Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some variable. But in your experiment, random sampling lead the value of that variable to be lower (on average) in the treated group, and enough lower that the difference is statistically significant. You'll correctly reject the null hypothesis of no difference (so won't have made a Type I error). But you'll conclude that the treatment reduces the value of the variable, when in fact it really (if you collected enough data) increases it. Type III errors are rare, as they only happen when random chance leads you to collect low values from the group that is really higher, and high values from the group that is really lower. (Definition from page 19 of Hsu).
Keywords: type 1 error, type 2 error, type 3 error error types