Please enable JavaScript to view this site.

Navigation: PRINCIPLES OF STATISTICS > Hypothesis testing and statistical significance

Advice: Avoid the concept of 'statistical significance' when possible

Scroll Prev Top Next More

The term "significant" is seductive and easy to misinterpret, because the statistical use of the word has a meaning entirely distinct from its usual meaning. Just because a difference is statistically significant does not mean that it is biologically or clinically important or interesting. Moreover, a result that is not statistically significant (in the first experiment) may turn out to be very important.

Using the conventional definition with alpha=0.05, a result is said to be statistically significant when a difference that large (or larger) would occur less than 5% of the time if the populations were, in fact, identical.

The entire construct of 'hypothesis testing' leading to a conclusion that a result is or is not 'statistically significant' makes sense in situations where you must make a firm decision based on the results of one P value. While this situation occurs in quality control, it doesn't really happen in other situations. Usually, as with clinical trials, decisions are made based on several kinds of evidence. In basic research, it is rare to make a decision based on one experiment.

If you do not need to make a decision based on one P value, then there is no need to declare a result "statistically significant" or not. Simply report the P value as a number, without using the term 'statistically significant'. Better, simply report the confidence interval, without a P value.

 

© 1995-2019 GraphPad Software, LLC. All rights reserved.