Harvey Motulsky's GraphPad Blog

Fitting dose-response curve when X is dose, rather than log(dose).

2009-11-18T10:59:16-08:00

The dose-response equations built-in to Prism all assume that the X values are log(dose). You can either enter the data with X values as logarithms of doses, or use the Transform analysis to create a results table with the data arranged that way which can then be graphed and fit.

It is possible to fit data where X values are concentrations, rather than log(concentrations). It is necessary to adjust the equation accordingly.

Here is the equation built-in to Prism for fitting a variable slope (four-parameter) log(dose) response curve:

Y=Bottom + (Top-Bottom)/(1+10^((LogEC50-X)*HillSlope))

Here is the equation modified to expect X values to be concentrations, not logarithms, so the concentration does not need to be raised to the tenth power to antilog it:

Y=Bottom + (Top-Bottom)/(1+ (10^logEC50 /X)^HillSlope)

The equation still fits the logEC50, rather than the EC50. Why? Because the confidence intervals computed by Prism are always symmetrical around the parameter value. But the true uncertainty is only symmetrical on a logEC50 scale.

Download this Prism file to see how it works. The same data are fit and graphed twice.

In one version the X values are transformed to logarithms, and then fit to the equation built-in to Prism. Here the graph has a linear X axis, but the numbering is converted to powers-of-ten to show that the X values represent logarithms.
In the other version, the data are fit with the X values remaining as concentrations and fit to the equation showed above. Here the X axis is stretched to a logarithmic scale (top right of Format Graph dialog).

The two graphs look identical. The results of the two fits are identical.

Note that the second graph will only look good in Prism 5, which is smart about plotting curves on axes stretched to a logarithmic scale. Prism 4 was not smart about this, and the resulting curve looks very choppy.

Why is the HillSlope applied to the EC50 as well as the X values?

Why doesn't Prism report the standard error of the EC50?

GraphPad programs and OSX 10.6 (Snow Leopard)

2009-08-31T08:35:40-08:00

Apple released a new version of OSX, 10.6 Snow Leopard, on Aug. 28 2009.

Prism 5

We know of one problem using Prism 5.0b on Snow Leopard: Fill patterns don't render well. We recommend that you use solid fills for bars, and simply avoid fill patterns altogether if you use Snow Leopard (and even otherwise, fill patterns are a hold over from the days of plotters, and solid fills look better). It is likely that Apple will fix this glitch in Snow Leopard. If they don't, we'll try to bypass the problem in release 5.0c.

Other minor glitches:

Black colors appear gray when the graph is exported to a pdf file using CMYK colors, and viewed in Preview. Choose RGB colors instead, and the pdfs look fine. Or export a tiff file with CMYK colors. Note that the pdf file is fine, but is just rendered incorrectly by the new version of Preview.
The Send-to-Powerpoint button and command don't work. Use Copy and Paste instead.
When running a Prism script, the script log is always empty.
Editing sheet names in the navigator looks ragged.
The slider on the info page separating info constants from notes looks corrupted.
Exporting to the PICT format doesn't work.
Exporting using the monochrome color model (to export colorful graphs as black and white) doesn't work.
If you save a Prism file as XML, its icon is blank.
One person found that the updater from 5.0a to 5.0b did not work under Snow Leopard. But the full 5.0b installer worked fine.

We will investigate these problems, and any others we discover or are told about, and fix in release 5.0c coming soon. Let us know if you encounter other problems.

InStat, StatMate, and Prism 4

InStat 3, StatMate 2 and Prism 4 use an older style of Mac programming. They run perfectly on current macs using an Intel chip, but do so by relying on Apple's Rosetta system. Apple created Rosetta so programs written for the earlier generation of Macs that use a PowerPC chip will also work on newer Intel Macs. This is truly amazing software that just works. You don't even know it is there.

With OSX 10.4 (Tiger) and 10.5 (Leopard), Rosetta was automatically installed and simply works when it is needed. You don't have to configure it, and won't even know when it is running. The only exception is that a few people have had problems after updating to OSX 10.5.6. This page from the Apple web site explains how to fix the problem, which requires running the 'combo update' rather than the 'incremental update' .

Rosetta is not automatically installed with OSX 10.6 (Snow Leopard). If you are updating to Snow Leopard and plan to run InStat 3, StatMate 2, or Prism 4, click the "Customize" button in the Mac OS X Snow Leopard installer and select the option to install Rosetta.

If you don't install Rosetta at the time you install Snow Leopard, or get a new Mac without it, InStat, StatMate and Prism 4 will still work just fine. The first time you run one of these programs under Snow Leopard, OSX detects that you need Rosetta and provides an easy way to install it. You only have to do this once. Rosetta will be installed from Apple's server if you are connected to the internet. Otherwise, you'll need to insert your Mac OS X Snow Leopard installation disc, open the Optional Installs folder, and double-click Optional Installs.

Guidelines for presenting statistics in published papers.

2009-08-25T09:08:30-08:00

Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publications is a lengthy document with guidelines for authors and publishers. But it has only one paragraph about statistics:

"Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results. When possible, quantify findings and present them with appropriate indicators of measurement error or uncertainty (such as confidence intervals). Avoid relying solely on statistical hypothesis testing, such as P values, which fail to convey important information about effect size. References for the design of the study and statistical methods should be to standard works when possible (with pages stated). Define statistical terms, abbreviations, and most symbols. Specify the computer software used."

These two papers give sensible guidelines for presenting statistical calculations and conclusions:

Curran-Everett and Benos. Guidelines for reporting statistics in journals published by the American Physiological Society. AJP - Gastrointestinal and Liver Physiology (2004) vol. 287 (2) pp. G307. Those authors later published a sequel, with additional comments. This sequel references a bunch of papers which critique the guidelines.

Ludbrook. The presentation of statistics in Clinical and Experimental Pharmacology and Physiology. Clin Exp Pharmacol Physiol (2008) vol. 35 (10) pp. 1271-4). Ludbrook has also self published a two-page set of guidelines for mathematical operators and statistical symbols.

These authors agree on two points (regarding style, not substance) that I was not aware of, so the GraphPad manuals and help screens (and my book Intuitive Biostatistics) have done differently:

They say that the standard error of the mean should be abbreviated as SE, rather than SEM.
They say that the mean and standard deviation should be written as mean (SD), rather than mean ± SD. if the mean is 11.2 and the standard deviation is 2.4, they suggest reporting 11.2 (2.4) rather than 11.2 ± 2.4. They recommend using that latter syntax only for standard errors, not standard deviations.

How does Prism compute and plot residuals from nonlinear regression?

2009-08-20T08:31:48-08:00

If you choose (or accept the default) standard weighting, then the residuals are the difference between the actual Y value you entered and the Y value predicted by the model. If the data point is above the curve, the residual is positive. If the data point is below the curve, the residual is negative. Least-squares regression works to minimize the sum of the squares of these residuals.

If you choose another weighting scheme, Prism 5 adjusts the definition of the residuals accordingly. The residual that Prism tabulates and plots equals the residual defined in the prior paragraph, divided by the weighting factor. The most common common alternative weighting is "Weight by 1/Y² (minimize relative distances squared)". In this case, the residual is defined to be the distance of the point from the curve divided by the Y value of the curve. Weighted nonlinear regression minimizes the sum of these residuals squared.

Note the ambiguity in defining weighting. The Prism dialog gives the choice to weight by 1/Y². This means that the squared residual is divided by Y². The weighted residual is defined as the residual divided by Y. Prism minimizes the sum of the squares of these weighted residuals.

Earlier versions of Prism (up to Prism 4) always plotted basic unweighted residuals, even if you chose to weight the points unequally.

When performing linear regression, Prism does not offer weighting so the residuals are always unweighted residuals as defined in the first paragraph above.

How Prism computes R² with weighted nonlinear regression.

How weighted nonlinear regression works.

Adjusted P values as part of multiple comparisons.

2009-08-17T18:47:45-08:00

Many people ask why multiple comparisons tests following one-way (or two-way) ANOVA can't report individual P values for each comparison. When you correct for multiple comparisons, it really doesn't make much sense to talk about individual P values. All you can do is divide the comparisons into two groups -- statistically significant and not -- at some defined significance level (usually 5%) that applies to the entire family of comparisons.

However SAS reports adjusted P values, and these are explained in the book by Westfall (citation below).

The idea is pretty simple. There is nothing special about 0.05 or 0.01... You can set the significance level to any probability you want. The adjusted P value is the smallest probability at which a particular comparison will be declared statistically significant (as part of the multiple comparison testing). Each comparison will have a unique adjusted P value. But these P values are computed from all the comparisons, and really can't be interpreted for just one comparison.

Computing the adjusted P value is trivial for Bonferroni multiple comparison tests. It is harder for Tukey and Dunnett tests, as it requires computing critical values beyond those that have been tabulated.

We are considering including adjusted P values as part of the reporting of one-way ANOVA with Tukey, Bonferroni or Dunnett multiple comparison tests. I hesitate to do this (the programming is far from trivial) because I don't really know how to explain how to interpret the results, and fear that they will be misinterpreted.

I don't know of any program except SAS that computes adjusted P values, and don't know of any book except Westfall that explains them. They do not seem to be mainstream.

Please let us know if you would like to see adjusted P values in future versions of Prism, and explain why.

	Multiple Comparisons and Multiple Tests (Text and Workbook Set)
	by Peter H. Westfall, Randall D. Tobias, Dror Rom
	IBSN:1580258336. List price:$62.32
	Buy from amazon.com for $50.18

Bug with Fisher's Exact test in Prism 5.02 and 5.0b

2009-08-10T11:31:45-08:00

Prism 5.02 (Windows) and 5.0b (Mac) included a fix to a trivial bug in Fisher's exact test (when the two groups are identical, the P value should be 1.00 but earlier versions of Prism sometimes reported P values slightly greater than 1.0). Unfortunately, that fix introduced a new bug that occurs only when:

You are using Prism 5.02 (Windows) or 5.0b (Mac). Earlier versions did not have this bug. Neither does InStat 3.0 or 3.1.
You have entered a symmetrical contingency table. A table is symmetrical when either the two row totals are identical, or the two column totals are identical.
You have chosen a two-tail (two-sided) P value. One-tail P values are computed correctly.

The result of the bug is that the P value will be too low in some, but not all cases. In many cases, the discrepancy is tiny and won't affect your conclusions. In other cases, the discrepancy is larger and may affect your conclusion.

Of course, we will fix the bug in the next release of Prism: 5.03 and 5.0c.

It is easy to determine whether you have encountered this bug, and to compute the correct two-tail P value. With symmetrical contingency tables, the two-tail P value is exactly twice the one-tail P value (that is not always true with contingency tables that are not symmetrical). Therefore, to bypass the bug, ask Prism to compute a one-tail (one-sided) P value. This is a choice in the Parameters dialog for analyzing contingency table. To compute a two-tail P value, simply double the one-tail P value.

More details and example.

GraphPad InStat 3.1 is available

2009-07-14T07:56:51-08:00

If you own a license for InStat 3, please update free to version 3.1. Updates are available for both Mac and Windows versions. The biggest change is that we've increased the size of the data table, which can now have 10,000 rows and 52 columns. Other changes are listed here. If you are not familiar with InStat, it is a very simple statistics program -- so simple, anyone can master it in just a few minutes. Learn more.

The dangers of smoothing.

2009-07-09T10:13:30-08:00

Matt Briggs has previously written about the dangers of smoothing here and here. The problem is simple: smoothing induces spurious correlations. His latest post points out that smoothing can make it appear that a prediction or forecast is far more accurate than it really is. I had a hard time following his argument, so I wrote a page which includes my own set of simulations and smoothing and my own explanations for why smoothing leads to problems. Here is another way to see the dangers of smoothing. Steve McIntyre at ClimateAudit.org showed how the S&P500 looks with and without smoothing. If you only saw the smoothed data, you'd get a very wrong impression about the state of the stock market.

The sum of two Gaussian distributions is not always bimodal.

2009-07-07T14:34:13-08:00

Is the distribution of height bimodal? It depends on who you include.

If you include both men and women, most people expect to see a bimodal distribution. In fact, it is not....Read more...

The distinction between confidence, prediction and tolerance intervals

2009-07-01T17:40:32-08:00

When you fit a parameter to a model, the accuracy or precision can be expressed as a confidence interval, a prediction interval or a tolerance interval. The three are quite distinct. The discussion below explains the three different intervals for the simple case of fitting a mean to a sample of data (assuming sampling from a Gaussian distribution). The same ideas can be applied to intervals for any best-fit parameter determined by regression.

Confidence intervals tell you about how well you have determined the mean. Assume that the data really are randomly sampled from a Gaussian distribution. If you do this many times, and calculate a confidence interval of the mean from each sample, you'd expect about 95 % of those intervals to include the true value of the population mean. The key point is that the confidence interval tells you about the likely location of the true population parameter.

Prediction intervals tell you where you can expect to see the next data point sampled. Assume that the data really are randomly sampled from a Gaussian distribution. Collect a sample of data and calculate a prediction interval. Then sample one more value from the population. If you do this many times, you'd expect that next value to lie within that prediction interval in 95% of the samples.The key point is that the prediction interval tells you about the distribution of values, not the uncertainty in determining the population mean.

Prediction intervals must account for both the uncertainty in knowing the value of the population mean, plus data scatter. So a prediction interval is always wider than a confidence interval.

Before moving on to tolerance intervals, let's define that word 'expect' used in defining a prediction interval. It means there is a 50% chance that you'd see the value within the interval in more than 95% of the samples, and a 50% chance that you'd see the value within the interval in less than 50% of the samples. What if you want to be 95% sure that the interval contains 95% of the values? Or 90% sure that the interval contains 99% of the values?

Those latter questions are answered by a tolerance interval. To compute, or understand, a tolerance interval you have to specify two different percentages. One expresses how sure you want to be, and the other expresses what fraction of the values the interval will contain. If you set the first value (how sure) to 50%, then a tolerance interval is the same as a prediction interval. If you set it to a higher value (say 90% or 99%) then the tolerance interval is wider.

The use and abuse of logarithmic axes

2009-06-26T10:48:55-08:00

Logarithmic axes are widely used by students and scientists, and are a frequent source of confusion and frustration. I wrote this 13 page article (table of contents below) to explain the uses and abuses of logarithmic axes. The article is written for users of GraphPad Prism, but almost all the information will be useful no matter how you make your graphs.

What is a logarithmic axis?

A logarithmic axis changes the scale of an axis.
Interpolating between log ticks
Why “logarithmic”?
Lingo
Other Bases

Logarithmic axes cannot contain zero or negative numbers

The logarithms of negative numbers and zero are simply not defined
A trick to plot zero on a logarithmic axis in Prism
Logarithmic axes on bar graphs are misleading

When to use a logarithmic axis

A logarithmic X axis is useful when the X values are logarithmically spaced
A logarithmic axis is useful for plotting ratios
A logarithmic axis linearizes compound interest and exponential growth
An exponential decay curve is linear on a logarithmic axis, but only when it decays to zero

Lognormal distributions

Plotting lognormal distributions on a logarithmic axis
The mean and geometric mean
Displaying variability on a lognormal distributions

Distinguish using a logarithmic axis from plotting logarithms

Regression fits the data, not the graph
Use antilog or powers-of-ten numbering when plotting values that are logs

Copy and paste tabular results from InStat to Excel. Aligned!

2009-06-26T06:57:25-08:00

InStat presents the results in text tables, using multiple spaces to align columns. It looks great within InStat, and prints fine. But if you copy the results and paste into Excel, it becomes a mess. The problem is that Excel only sees a paragraph, not a table, so it pastes everything into one column.

But it is easy to fix things so the results are tabular within Excel:

Select and copy a block of tabular results from InStat.
Go to an empty table in Excel. Put the insertion point in the top left cell and paste.
Select the first column in Excel. This column contains everything you pasted.
Drop the Data menu. In Excel 2007, go to the Data tab.
Choose the Text to Columns command. In Excel 2007, click the button with that name.
Choose Fixed Width.
Finish the wizard, and your data will be in aligned Excel columns.

I've tested this with Excel 2007 (Win) and 2008 (Mac). It ought to work with earlier versions as well. I was not able to find a similar command in Numbers (part of iWork for Mac).

We are testing InStat 3.1

2009-06-19T09:42:12-08:00

Our programmer have completed the work on InStat 3.1. The biggest change is that we've increased the maximum number of rows by a factor of ten, and doubled the maximum number of columns. Full list of changes. Contact me if you'd like to beta test InStat 3.1.

The essential concepts of statistics

2009-06-17T11:11:44-08:00

If you know twelve concepts about a given topic you will look like an expert to people who only know two or three.

Scott Adams, creator of Dilbert

When learning statistics, it is easy to get bogged down in the details, and lose track of the big picture. Here are the twelve most important concepts in statistical inference.

Statistics lets you make general conclusions from limited data.

The whole point of inferential statistics is to extrapolate from limited data to make a general conclusion. "Descriptive statistics" simply describes data without reaching any general conclusions. But the challenging and difficult aspects of statistics are all about reaching general conclusions from limited data.

Statistics is not intuitive.

The word ‘intuitive’ has two meanings. One meaning is “easy to use and understand.” That was my goal when I wrote Intuitive Biostatistics .The other meaning of 'intuitive' is “instinctive, or acting on what one feels to be true even without reason.” Using this definition, statistical reasoning is far from intuitive. When thinking about data, intuition often leads us astray. People frequently see patterns in random data and often jump to unwarranted conclusions. Statistical rigor is needed to make valid conclusions from data.

Statistical conclusions are always presented in terms of probability.

"Statistics means never having to say you are certain." If a statistical conclusion ever seems certain, you probably are misunderstanding something. The whole point of statistics is to quantify uncertainty.

All statistical tests are based on assumptions.

Every statistical inference is based on a list of assumptions. Don't try to interpret any statistical results until after you have reviewed that list. An assumption behind every statistical calculation is that the data were randomly sampled, or at least representative of, a larger population of values that could have been collected. If your data are not representative of a larger set of data you could have collected (but didn't), then statistical inference makes no sense.

Decisions about how to analyze data should be made in advance.

Analyzing data requires many decisions. Parametric or nonparametric test? Eliminate outliers or not? Transform the data first? Normalize to external control values? Adjust for covariates? Use weighting factors in regression? All these decisions (and more) should be part of experimental design. When decisions about statistical analysis are made after inspecting the data, it is too easy for statistical analysis to become a high-tech Ouja board -- a method to produce preordained results, rather an objective method of analyzing data.

A confidence interval quantifies precision, and is easy to interpret.

Say you've computed the mean of a set of values you've collected,or the proportion of subjects where some event happened. Those values describe the sample you've analyzed. But what about the overall population you sampled from? The true population mean (or proportion) might be higher, or it might be lower. The calculation of a 95% confidence interval takes into account sample size and scatter. Given a set of assumptions, you can be 95% sure that the confidence interval includes the true population value (which you could only know for sure by collecting an infinite amount of data). Of course, there is nothing special about 95% except tradition. Confidence itnervals can be computed for any degree of desired confidence. Amost all results -- proportions, relative risks, odds ratios, means, differences between means, slopes, rate constants... -- should be accompanied with a confidence interval.

A P values tests a null hypothesis, and is had to understand at first.

The logic of a P value seems strange at first. When testing whether two groups differ (different mean, different proportion, etc.), first hypothesize that the two populations are, in fact, identical. This is called the null hypothesis. Then ask: If thenull hypothesis were true, how unlikely would it be to randomly obtain samples where the difference is as large (or even larger) than actually observed? If the P value is large, your data are consistent with the null hypothesis. If the P value is small, there is only a small chance that random chance would have created as large a difference as actually observed. This makes you question whether the null hypothesis is true.

"Statistically significant" does not mean the effect is large or scientifically important.

If the P value is less than 0.05 (an arbitrary, but well accepted threshold), the results are deemed to be statistically significant. That phrase sounds so definitive. But all it means is that, by chance alone, the difference (or association or correlation..) you observed (or one even larger) would happen less than 5% of the time. That's it. A tiny effect that is scientifically or clinically trivial can be statistically significant (especially with large samples). That conclusion can also be wrong, as you'll reach a conclusion that results are statistically significant 5% of the time just by chance.

"Not statistically significant" does not mean there is no difference (or association . . .).

If a difference is not statistically significant, you can conclude that the observed results are not inconsistent with the null hypothesis. Note the double negative. You cannot conclude that the null hypothesis is true. It is quite possible that the null hypothesis is false, and that there really is a difference between the populations. This is especially a problem with small sample sizes. It makes sense to define a result as being statistically significant or not statistically significant when you need to make a decision based on this one result. Othewise, the concept of statistical significance adds little to data analysis.

Multiple comparisons make it hard to interpret statistical results.

When many hypotheses are tested at once, the problem of multiple comparisons makes it very easy to be fooled. If 5% of tests will be "statistically significant" by chance, you expect lots of statistically significant results if you test many hypotheses. Special methods can be used to reduce the problem of finding false, but statistically significant, results, but these methods also make it harder to find true effects. Multiple comparisons can be insidious. It is only possible to correclty interpret statistical analyses when all analyses are planned, and all planned analyses are conducted and reported. However, these simple rules are widely broken.

Correlation does not mean causation.

A statistically significant correlation or association between two variables may indicate that one variable causes the other. But it may just mean that both are influenced by a third variable. Or it may be a coincidence.

Published statistics tend to be optimistic.

Confidence intervals vs. confidence bands for survival curves

2009-06-11T10:45:57-08:00

When Prism computes survival curves, it can also compute the 95% confidence interval at each time point (using two alternative methods). The methods are approximations, but can be interpreted like any confidence interval. You know the observed survival percentage at a certain time in your study, and can be 95% confident (given a set of assumptions) that the confidence interval contains the true population value (which you could only know for sure if you had an infinite amount of data).

When these confidence intervals are plotted as error bars (left graph below) there is no problem. Prism can also connect the ends of the error bars, and create a shaded region (right graph below). This survival curve plots the survival of a sample of only seven people, so the confidence intervals are very wide. Prism file.

The shaded region looks like the confidence bands computed by linear and nonlinear regression, so it is tempting to interpret these regions as confidence bands. But it is not correct to say that you can be 95% certain that these bands contain the entire survival curve. It is only correct to say that at any time point, there is a 95% chance that the interval contains the true percentage survival. The true survival curve (which you can't know) may be within the confidence intervals at some time points and outside the confidence intervals at other time points.

It is possible (but not with Prism) to compute true confidence bands for survival curves, and these are wider than the confidence intervals shown above. The graph below (from Coviello) shows the survival curve in black (the sample was large, so the steps are small), the confidence limits in green, and the confidence bands in red. Confidence bands that are 95% certain to contain the entire survival curve (red) are wider than the confidence intervals for individual time points.

(Thanks to Joe Felsenstein for pointing out the distinction between confidence intervals and confidence bands in survival curves. )