Calculating a Z-factor to assess the quality of a screening assay.

When developing (or assessing) an assay to test the effectiveness of various drugs, you want to quantify how well the assay works. One way to do this is via the signal to noise ratio, but this doesn't really capture what you want to know. Zhang and colleagues developed a method to quantify the quality of an assay (1).

 

 

The figure above  (Figure 4 from Zhang, with a few extra labels) defines the separation band. The horizontal axis shows the value determined by the assay. The vertical axis shows how commonly each value occurs. The graph shows data for both negative and positive controls. The idea is simple:

 

• Virtually all the background values will be less than a threshold defined as the mean of the background values plus three times the standard deviation of those values. If the values come from a Gaussian distribution, you expect  99.86% of the values to be less than that threshold  and 0.14% of the values to be greater than that threshold. 
• Similarly, you expect virtually all of the true "hits" to have values greater than a threshold set by the mean of the positive controls minus three times the standard deviation of those values. 
• The separation band is the difference between those two thresholds.
• The dynamic range of the assay is defined as the difference between the means of the negative and positive controls.
• Comparing the lengths of the separation band and dynamic rangle tells you about how well the assay works. 

Zhang and colleagues (1) defined Z as the result of the following calculations. 

 

1. Compute the threshold value for negative controls as the mean signal of the negative controls plus three times their standard deviation.
2. Compute the threshold value for positive controls as the mean signal of the positive controls minus three times their standard deviation.
3. Compute the  difference between the two thresholds and call it the 'separation band' of the assay, S. If the threshold computed in step 1 is less than the one computed in step 2, then this difference is positive. Otherwise, this difference will have a negative value.
4. Compute the absolute value of the difference between the two means and call it the 'dynamic range' of the assay, R.
5. Compute the Z as   S/R

To interpret the Z-factor , use these guidelines (direct from Zhang's paper).

• A Z-factor  of 1, ideal. An assay can never have a Z-factor of 1.00000.  This value is approached when you have a huge dynamic range with tiny standard deviations. In this situation, the separation band is almost as long as the dynamic range. Z-factors  can never be greater than 1.0.
• A Z-factor  between 0.5 and 1.0 is an excellent assay.
• A Z-factor  between 0 and 0.5 is marginal.
• A Z-factor  less than 0 means that the signal from the positive and negative controls could overlap, making the assay not very useful or screening purposes.

Note that the use of the variable Z  here has absolutely nothing to do with  the use of z to describe how far a value is from the mean as the z ratio, which is the number of standard deviations away from the mean. All statistics books use z in this context, which has nothing to do with the Z-factor used to assess a screening assay.

Also note that the Z factor is computed based on a fairly arbitrary equation. Why compute the thresholds as three times the respective standard deviations rather than use a factor of two or four or any other factor? It's arbitrary. But experience has shown that this Z factor is a useful way to describe an assay, and it has become standard.

No GraphPad program computes the Z-factor. But you can use the Row or Column statistics analyses in Prism to compute the means and SDs. Once you have those values, computing the Z-factor is easy.

An alternative way to quantify the difference between means compared to the SD of the two groups is to compute R².  This is not commonly done as part of t test calculations, but is part of the output of GraphPad Prism.

----

(1) Zhang JH, Chung TD, Oldenburg KR, A Simple Statistical Parameter for Use in Evaluation and Validation of High Throughput Screening Assays. J Biomol Screen. 1999;4(2):67-73.

 

[This page was rewritten in Sept. 2010 to include the figure and to better explain the calculations.]