July 16, 2008

Tips to make Prism 5 files much smaller!

Prism 5 files can get large, but we offer two ways to make them smaller. 

The best  method is to save the projects in the new PZFX file format, instead of the PZF format. This is a choice in the File... Save As dialog. Change the default in the File&Printer tab of Prism's Preferences dialog.

With this format, the data and info tables are in plain text XML, but the rest of the file is greatly compressed. Overall, PZFX files tend to be MUCH smaller than PZF files. Typically, a PZFX file is less than 10% the size of the corresponding PZF file.

If you only use Prism 5,  there is no reason to avoid PZFX files. Not only are they smaller, but also are a bit more secure. All the data are stored in the first part of the file in plain text that could be recovered with a text editor, even if the file is damaged. The only drawback of choosing the PZFX format, is that these files can only be opened by Prism 5, and not by Prism 4. In contrast, PZF files can be opened by either Prism 4 or 5.

If you want to use PZF files (for compatibility with Prism 4), here is a trick to reduce the file size a bit. Go to the Preferences dialog, File tab, and check the option to "Save compact". With this option checked, Prism won't save the results of analyses, but rather recomputes the results when it opens the file. The difference in file size depends on how many analyses your file contains. 

July 15, 2008
How to compare two (or more) means when the groups have different standard deviations.

The unpaired t test depends on the assumption that the samples come from populations that have identical standard deviations (and thus identical variances). One-way ANOVA makes the same assumption.

Prism tests this assumption using an F test (to compare the variance of two groups) and Barlett’s test (three or more groups).

The P value answers the question:

If the populations really had identical standard deviations, what is the chance of observing as large a discrepancy among sample standard deviations as occurred in the data (or an even larger discrepancy)?

Note: Don’t mix up the P value testing for equality of the standard deviations of the groups with the P value testing for equality of the means. That latter P value is the one that answers the question you most likely were thinking about when you chose the t test or one-way ANOVA.

If the P value is large (>0.05) you conclude that there is no evidence that the standard deviations differ. If the P value is small, you conclude that the data are likely to be sampled from populations with different standard deviations.

Then what? There are four answers.

  • Conclude that the populations are different, that the treatment had an effect. In many experimental contexts, the finding of different variances is as important as the finding of different means. If the standard deviations are truly different, then the populations are different regardless of what the t test or ANOVA concludes about differences between the means. This may be the most important conclusion from the experiment.
  • Transform your data in an attempt to equalize the standard deviations, and then run the t test or ANOVA on the transformed results. Logs are especially useful. If the data are sampled from a lognormal distribution, then the logarithms of the data will follow a Gaussian distribution.
  • Ignore the result. With equal, or nearly equal, sample size (and moderately large samples), the assumption of equal standard deviations is not a crucial assumption. The t test and one-way ANOVA work pretty well even with unequal standard deviations. In other words, the t test and one-way ANOVA are remarkably robust to violations of that assumption so long as the sample size isn’t tiny and the sample sizes aren’t far apart.
  • Go back and rerun the t test, checking the option to do the modified Welch t test that allows for unequal variance. While this sounds sensible, Moser and Stevens (Amer. Statist. 46:19-21, 1992) have shown that it is not a good idea to first look at the F test to compare variances, and then switch to the modified (Welch modification to allow for different variances) t test when the P value is less than 0.05. The Welch test must be specified as part of the experimental design. (Modifications of one-way ANOVA are also available that do not assume equal standard deviations.)

Note that switching to a nonparametric test is not an appropriate approach. If the groups are sampled from populations with distinct standard deviations, then the nonparametric tests simply test whether the distributions are different. They do not test whether the medians differ

July 8, 2008

Asymmetrical (five-parameter) logistic dose-response curves

The standard log(dose) vs. response curve is defined by the bottom, top, EC50, and slope. In this curve, the top and bottom parts are mirror images of each other -- the curve is symmetrical.

Some log(dose) vs. response curves are not symmetrical. This can be modeled by including a fifth parameter that describes the asymmetry of the curve. The standard curve is sometimes called a four-parameter logistic model, so the asymmetrical curve is called a five parameter logistical model. 

Of course, an equation should not be referred to by its number of parameters. Some authors assume that any nonspecific signal is already subtracted off, so present the equations in a form where Bottom is defined to be zero and doesn't appear in the equation. Then the symmetrical variable slope equation has three (not four) parameters, and the asymmetrical form has four (not five) parameters.

Asymmetrical dose-response curves can be described by several equations. Prism uses the RIchards version (from Giraldo et. al.), which  is built-in to the 'Dose-response -- Special' group of equations in Prism 5. This is also called the generalized Hill equation.

LogXb = LogEC50 + (1/HillSlope)*Log((2^(1/S))-1)
Numerator = Top - Bottom
Denominator = (1+10^((LogXb-X)*HillSlope))^S
Y = Bottom + (Numerator/Denominator)

This equation assumes that X has been entered as (or transformed to) the logarithm of concentration, and that Y is response in any convenient units. 

S is the asymmetry parameter. If s=1.0, then this is the same as the four parameter equation. When s is not 1.0, the curve will be asymmetrical. S must be greater than zero, but can be less than, or greater than, 1.0. 

Top and Bottom are the Y values at the top and bottom plateaus of the curve. If you have normalized the data, you may want to constrain these values to 100 and 0. 

The equation above fits the logEC50, which is the X value when Y is half-way between the Top and Bottom plateaus. It is in the same units as the X values -- the logarithm of concentration.  Note that the logEC50 is not the same as the inflection point. The first line in the equation computes the inflection point from the logEC50, HillSlope, and S. Like the logEC50, this inflection point (called logXb) is in the same scale as the X values (the logarithm of concentration). 

 

If your goal is to obtain meaningful best-fit parameters, then you'll need lots of high quality data. It is very hard to fit both slope and asymmetry with tight confidence intervals. If your goal is just to interpolate unknowns from a standard curve, the width of the confidence intervals of the parameters doesn't really matter. What you want is a curve that follows the data, and in some cases an asymmetrical five parameter model does so better than a four parameter model. 

 

Other formulations of asymmetrical dose-response curves have been developed. For example, Ricketts and Head developed a model for use in baroreflex studies. 

Bindslev has written a lengthy on-line text, Drug-Acceptor Interactions.  Chapter 10,  Hill in Hell discusses many models of dose-response curves, including asymmetrical ones. 

June 8, 2008

Centered polynomial regression

The standard polynomial models look like this:

Y= B0 + B1*X +B2*X^2


More terms are included with the higher order equations. There are two problems with polynomial fits:

  • When the X values are large, and start well above zero (for example, when  X is calendar year from 1980 to 2005), taking the very large X values to large powers can lead to math overflow. Even if the program doesn't report any math error, the results can be inaccurate. Some coefficients will be positive and some negative, so the value of Y depends on subtracting huge numbers from other huge numbers, leading to imprecise results.
  • Even when the X values are not large, the parameters of the model are intertwined, so have high covariance and dependency. This results in large standard errors , and wide confidence intervals, and confidence or prediction bands. In many cases, this problem is severe enough that Prism reports that the results are 'ambiguous' and so doesn't report confidence intervals for all the parameters and can't graph confidence bands.


Both problems go away, when the X values are centered. The idea of centering is to subtract the mean X from all X values before fitting the model. This can be done as part of nonlinear regression,  using this model:

             XC = X - Xmean

Y= B0 + B1*XC +B2*XC^2


Here XC is the centered X value, equal to the X value minus Xmean, which  is the mean of all X values. Xmean needs to be a constant, and not a parameter that Prism tries to fit.  Of course, you can include more terms in the definition of Y. 

Fitting the centered model leads to exactly the same curve (unless the regular  approach led to math errors). Accordingly, the sum-of-squares is the same, which means that the results of model comparisons are identical.

However, the centered model has reparameterized the equation. The parameters have different meanings, so have different best-fit values (except the last parameter which is the same), different standard errors and confidence intervals , smaller covariances and dependencies,  and narrower confidence/prediction bands.

With Prism 5.02 and 5.0b, you will able to constrain XMean to be a column constant equal to the mean X values. Until these versions are released, you'll need to compute the mean X value manually, and then constrain XMean to a constant value (and enter the mean X you already computed).

Here is a Prism file that demonstrates centered polynomial fitting. Open it, go to one of the fits, change parameters and then then "edit" the equation without changing anything. This will place the equation into your user-defined equation list. 

June 6, 2008

 Are outlier tests useful when data come from a distribution that is not Gaussian?

No.

Most outlier tests are based on the assumption that the data, except the potential outier(s), come from a Gaussian distribution. If the distribution is not Gaussian, outlier tests are misleading. Here is an example.  Grubbs outlier test found an outlier in three of these four data sets.

But these data are not sampled from a Gaussian distribution with an outlier. Rather they are sampled from a lognormal distribution. Transform all the values to their logarithms, and the distribution becomes Gaussian:

The apparent outliers are gone. Grubb's test finds no ouliters.  The extreme points only appeared to be outliers because extreme values are common in a lognormal distribution but rare in a Gaussian distribution. If you don’t realize the distribution was lognormal, an outlier test would be very misleading.

 

Removing all values that are too big or too small.

 When analyzing data, sometimes you want to graph or analyze only a portion of the values, and remove any values that are higher (or lower) than some threshold. You can do this with a user-defined Prism transform. Here is a transform that removes any data with Y greater than 100:

   Y=IF(Y>100, 0/0, Y)

That transforms any values greater than 100 to 0/0 which is undefined, so becomes blank in the results table. The other values get transformed to equal Y (no change). 

Here is a transform that removes any data with Y greater than 100 or less than 10. 

  Y=IF(Y>100, 0/0, IF(Y<10, 0/0,Y))

This simply nests two IF functions in the transform. 

To enter a user defined tranform, go to a data table, click analyze, and choose Transform. At the top of the dialog, choose User-defined Y transforms. On the new dialog, click Add to create a new transform. 

Of course, you could create an X transform and use similar syntax to remove rows where X is too high or too low (or meets some other criterion). 

June 4, 2008
How to make the right and left Y axes look different

When you create a graph with two Y axes, Prism always creates them with the same length and the same color.

To make the lengths appear different:

  1. Use the rectangle tool to draw a rectangle over the part of the axis you don't want to see
  2. Make it white (or whatever your page background is) with a solid fill. Then that axis appears to be shorter.

To give the axes different colors:

  1. Click on one of the axes to select.
  2. Drop the Change menu, and choose Selected Object
  3. Change the color.

To give the axis numbering a different color or font:

  1. Click on one of the axes to select it.
  2. Use toolbar buttons to change the color or font used to number that axis.

To only have one axis but put in on the right side of the graph:

The first axis created is called the 'left Y axis', but in fact it does not need to be placed on the left side of the graph. It can be anywhere. To put this  axis on the right side of a graph:

  1. Double click on an axis to bring up Format Axis dialog
  2. Go to the first tab, Frame and Origin.
  3. Set the Origin to the lower right. 

To delete the right Y axis:

  1. Double-click on it to bring up the Format Axis dialog, 
  2. Make sure you are on the right Y axis tab.
  3. Drop down the list labeled 'Gaps and direction'.
  4. Choose: No right Y axis. 

 

June 1, 2008

Plotting t, z, F, or chi-square distributions with Prism.

GraphPad Prism can generate probability distributions. This demonstrates Prism's ability to plot functions from user-defined functions, and also the use of hooking info constants to analyses.

Download this Prism 5 file to generate and plot the graphs shown below. 
 

z, t, F, chisquare distributions

 

In each case, the simulation generates two (or three) data sets. The first (A) data set plots the entire curve. The second (and third) data sets only plot values where X is greater than (less than) a specified cutoff value. This second (and third) data set are plotted with area fill to shade the tails of the distributions. Remove data set B or C from the graph if you only want to shade one tail. 

Change the numbers of degrees of freedom and the cutoff values (for shading) in the Info sheet. This demonstrates how values entered into an info sheet can be 'hooked' to constants used in analyses.

 Graphing a Binomial or Poisson distribution with Prism. 

 Prism can graph a Binomial or Poisson distribution. Download the file that generated this pair of graphs. 

 

To modify this file, change the value of lamda (for Poission) or the probability, n, and cutoff (Binomial) in the Info sheet. Enter new values there, and the graph updates. This is a good example of the usefulness of hooking an info constant to an analysis.

If you want to recreate graphs like these, keep in mind these points:

  •  As its name suggests, the analysis 'Create a Family  of Theoretical Curves' is usually used to create curves, not bar graphs. When you choose the range of X values, specify the appropriate number of 'line segments' (points) so that the X interval equals 1.0. The binomial example on the left created 16 'line segments' starting at X=0 and ending at X=15. The Poisson on the right created 13 'segments' with X starting at 0 and ending at 13.
  • The analysis will create a set of line segments (an attempt to create a curve). Click the change type of graph button, or drop the Change menu and choose Graph Type. Then choose the Grouped tab, and then choose interleaved bars. 
  • The binomial example on the right has two data sets. You don't want them plotted interleaved, as selected in the previous step. Double click to bring up Format Graph, then go to the middle tab, and choose to superimpose the second data set on the first (rather than interleave). And assign it a different color. 

 

May 25, 2008

P values. One-tail or  two-tail ?

When comparing two groups, you must distinguish between one- and two-tail P values. Some books refer to one-sided and two-sided P values, which mean the same thing.

What does one-sided mean?

It is easiest to understand the distinction in context. So let’s imagine that you are comparing the mean of two groups (with an unpaired t test). Both one- and two-tail P values are based on the same null hypothesis, that two populations really are the same and that an observed discrepancy between sample means is due to chance.

A two-tailed P value answers this question:

Assuming the null hypothesis is true, what is the chance that randomly selected samples would have means as far apart as (or further than) you observed in this experiment with either group having the larger mean?


To interpret a one-tail P value, you must predict which group will have the larger mean before collecting any data. The one-tail P value answers this question:

Assuming the null hypothesis is true, what is the chance that randomly selected samples would have means as far apart as (or further than) observed in this experiment with the specified group having the larger mean?


If the observed difference went in the direction predicted by the experimental hypothesis, the one-tailed P value is half the two-tailed P value (with most, but not quite all, statistical tests).

When is it appropriate to use a one-sided P value?

A one-tailed test is appropriate when previous data, physical limitations, or common sense tells you that the difference, if any, can only go in one direction. You should only choose a one-tail P value when both of the following are true.

  • You predicted which group will have the larger mean (or proportion) before you collected any data.
  • If the other group had ended up with the larger mean – even if it is quite a bit larger – you would have attributed that difference to chance and called the difference 'not statistically significant'.

Here is an example in which you might appropriately choose a one-tailed P value: You are testing whether a new antibiotic impairs renal function, as measured by serum creatinine. Many antibiotics poison kidney cells, resulting in reduced glomerular filtration and increased serum creatinine. As far as I know, no antibiotic is known to decrease serum creatinine, and it is hard to imagine a mechanism by which an antibiotic would increase the glomerular filtration rate. Before collecting any data, you can state that there are two possibilities: Either the drug will not change the mean serum creatinine of the population, or it will increase the mean serum creatinine in the population. You consider it impossible that the drug will truly decrease mean serum creatinine of the population and plan to attribute any observed decrease to random sampling. Accordingly, it makes sense to calculate a one-tailed P value. In this example, a two-tailed P value tests the null hypothesis that the drug does not alter the creatinine level; a one-tailed P value tests the null hypothesis that the drug does not increase the creatinine level.

The issue in choosing between one- and two-tailed P values is not whether or not you expect a difference to exist. If you already knew whether or not there was a difference, there is no reason to collect the data. Rather, the issue is whether the direction of a difference (if there is one) can only go one way. You should only use a one-tailed P value when you can state with certainty (and before collecting any data) that in the overall populations there either is no difference or there is a difference in a specified direction. If your data end up showing a difference in the “wrong” direction, you should be willing to attribute that difference to random sampling without even considering the notion that the measured difference might reflect a true difference in the overall populations. If a difference in the “wrong” direction would intrigue you (even a little), you should calculate a two-tailed P value.

Recommendation
I recommend using only two-tailed P values for the following reasons:

  • The relationship between P values and confidence intervals is more straightforward with two-tailed P values.
  • Two-tailed P values are larger (more conservative). Since many experiments do not completely comply with all the assumptions on which the statistical calculations are based, many P values are smaller than they ought to be. Using the larger two-tailed P value partially corrects for this.
  • Some tests compare three or more groups, which makes the concept of tails inappropriate (more precisely, the P value has more than two tails). A two-tailed P value is more consistent with P values reported by these tests.
  • Choosing one-tailed P values can put you in awkward situations. If you decided to calculate a one-tailed P value, what would you do if you observed a large difference in the opposite direction to the experimental hypothesis? To be honest, you should state that the P value is large and you found “no significant difference.” But most people would find this hard. Instead, they’d be tempted to switch to a two-tailed P value, or stick with a one-tailed P value, but change the direction of the hypothesis. You avoid this temptation by choosing two-tailed P values in the first place.


When interpreting published P values, note whether they are calculated for one or two tails. If the author didn’t say, the result is somewhat ambiguous.

Common misunderstandings about P values.

Kline (see book listing below) lists commonly believed fallacies about P values, which I summarize here:
 
Fallacy: P value is the probability that the result was due to sampling error
The P value is computed assuming the null hypothesis is true. In other words,  the P value is computed based on the assumption that the difference was due to sampling error. Therefore the P value cannot tell you the probability that the result is due to sampling error. 

Fallacy: The P value Is the probability that the null hypothesis is true
Nope. The P value is computed assuming that the null hypothesis is true, so cannot be the probability that it is true.

Fallacy: 1-P is the probability that the alternative hypothesis is true
If the P value is 0.03, it is very tempting to think: If there is only a 3% probability that my difference would have been caused by random chance, then there must be a 97% probability that it was caused by a real difference. But this is wrong!

What you can say is that if the null hypothesis were true, then 97% of experiments would lead to a difference smaller than the one you observed, and 3% of experiments would lead to a difference as large or larger than the one you observed.

Calculation of a P value is predicated on the assumption that the null hypothesis is correct. P values cannot tell you whether this assumption is correct. P value tells you how rarely you would observe a difference as larger or larger than the one you observed if the null hypothesis were true.

The question that the scientist must answer is whether the result is so unlikely that the null hypothesis should be discarded.

Fallacy: 1-P is the probability that the results will hold up when the experiment is repeated
If the P value is 0.03, it is tempting to think that this means there is a 97% chance of getting ‘similar’ results on a repeated experiment. Not so.

Fallacy: A high P value proves that the null hypothesis is true.
No. A high P value means that if the null hypothesis were true, it would not be surprising to observe the treatment effect seen in this experiment. But that does not prove the null hypothesis is true.

Reference: RB Kline, Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research, 2004.

The Mann-Whitney test doesn't really compare medians.

You'll sometimes read that the Mann-Whitney test compares the medians of two groups. But this is not precisely correct. 

Consider this example:

 

The graph shows each value obtained from control and treated subjects. The two-tail P value from the Mann-Whitney test is 0.0288, so you conclude that there is a statistically significant difference between the groups. But the two medians, shown by the horizontal lines, are identical. The Mann-Whitney test compared the distributions of ranks, which is quite different in the two groups even though the medians are the same.

It is not correct, however, to say that the Mann-Whitney test asks whether the two groups come from populations with different distributions. The two groups in the graph below clearly come from different distributions, but the P value from the Mann-Whitney test is high (0.46).

The Mann-Whitney test compares sums of ranks -- it does not compare medians and does not compare distributions.

The Mann-Whiteny test is a comparison of medians only when you assume that the distributions of the two populations have the same shape, even if they are shifted (have different medians). If you accept this assumption, then a small P value from a Mann-Whitney test leads you to conclude that the difference between medians is statistically signficant.

More generally, the P value answers this question: What is the chance that a randomly selected value from the population with the larger median is greater than than a randomly selected value from the other population?

 

May 20, 2008

Before-after graphs with different colors for different subjects.

When you enter data on a column table and choose a before-after graph, Prism plots all the symbols the same way. You can choose different colors or shapes for "before" than for "after" (which is not helpful). And you can right click on each symbol and change its color (and that of the connecting line). But this approach would be very tedious. .

Prism 5 can, in fact, create before-after graphs with multiple colors for different subjects. The trick is to enter the data enter the data on a Grouped table. Follow these steps or examine this Prism file.

1. Create a Grouped table.

 

 

Choose the appropriate number of "replicates" (subjects) for your data. Be sure to choose to plot each replicate, and to connect each replicate.

2. Enter the data.

Note that the arrangement of data is different than with a column table. The before-after pairs are stacked into subcolumns.

This table has two rows, because it plots just before and after. If you had more time points, add more rows.

This table has two data sets, male and female, because we want symbols with two different appearances. Use as many data sets as you want. If you want each subject to have its own appearance, create a table with no subcolumns, and enter each subject into its own data set.

3. Polish the graph.

 


Also see this related example for creating column scatter graphs with multiple symbol colors.

How to turn off automatic snapping.

When you move a text object in Prism 5, it will automatically snap into alignment with bars on graphs, groups of bars, the center of the page, and other text objects. This almost always is a great feature, as it lets you quickly move text to an appropriate spot. But sometimes, you may find that the automatic snapping prevents you from fine-tuning a graph.

There are two ways to work around the automatic snapping.

If you just want to nudge the text object a tiny bit, select it, and then use the arrow keys on the keyboard. Each click will move the object one pixel.

If you want to move the object with the mouse without snapping, hold the ALT key while dragging.

On the Mac, the Alt key is also called the Option key. Be sure to first start dragging and then hold down the ALT key. If you click the ALT key first, and then start dragging with Prism Mac, you will have created a duplicate copy of the object.

The Standard Addition Method for determing concentrations.

Prism can easily interpolate from a linear or nonlinear standard curve. You perform the assay at a number of known concentrations, fit a line or curve, and interpolate the uknown values.

But there is a problem with interpolating from a standard curve. The results can be incorrect when the unknown sample are contaminated with other substances that alter the assay. This is known as the 'matrix effect problem'.

The Standard Addition Method is a way to bypass this problem. You don't need to perform the assay with known concentrations of substance. Instead you add various known concentrations (including zero) of known substance to a constant amount of the unknown. This ensures that all the samples have the same amount of unknown, including any substances that interfere with the assay.

Fit the data with linear regression. The value you want to know is how much of the known substance has to be added to double the signal. There is an easier, somewhat trickier, way to find out: Extrapolate the line down to Y=0.  One of the parameters that Prism reports is the X intercept, which will be negative. Take the absolute value, and that is the concentration of the unknown substance. The confidence inerval for the X intercept gives you the confidence interval for the concentration of the uknown. Simply multiply both confidence limits by -1.

To plot the data in Prism, you'll want to extend the linear regression line to start at an X value equal to the X intercept (a choice in the Linear regression parameters dialog). You may also want to move the origin to the lower left, a choice on the first tab of the Format Axis dialog. Here is a graph created with this Prism file.