Sensitivity and specificity at various thresholds

The list of thresholds is taken by sorting all the values in both groups (patients and controls) and averaging adjacent values in that sorted list. So each threshold value is midway between two values in the data.

Each sensitivity is the fraction of values in the patient group that are above the threshold. The specificity is the fraction of values in the control group that are below the threshold. Each confidence intervals is computed from the observed proportion by the Clopper method (1), without any correction for multiple comparisons.

Area under the ROC curve

Prism uses the same method it uses for the Area Under Curve analysis.

SE of the area

Prism uses the method of Hanley (1), which uses the equation below where A is the area, na and nn are the number of abnormals (patients) and normals (controls).

Q1 is the probability that two randomly chosen patients will both get a more positive test result than a randomly chosen control which is approximately A/(2-A).

Q2 is the probability that one randomly chosen patient will get a more positive test result than two randomly chosen controls which is approximately 2A*A/(1+A).

Prism actually computes Q1 and Q1 using a more complicated equation.

P value

When computing the P value, Prism computes the SE differently, assuming that the area is really 0.5 (the null hypothesis). This simplifies the equation to

It then computes a z ratio using the equation below, and determines the P value from the normal distribution (two-tail).

Reference

1. C. J. Clopper and E. S. Pearson, The use of confidence or fiducial limits illustrated in the case of the binomial, Biometrika 1934 26: 404-413.

2. Hanley JA, McNeil BJ. The meaning and use of the area under the Receiver Operating Characteristic (ROC) curve,  Radiology 1982 143 29-36