Receiver Operating Characteristic Curves and Uncertainty of Measurement

Aristides T. Hatjimihail, MD, PhD
 ath@hcsl.com

Hellenic Complex Systems Laboratory, Kostis Palamas 21, 66131 Drama, Greece.

Initialization Code

roc[x_,μD_,μH_,σD_,σH_,u_]:=
1-CDF[NormalDistribution[μD,(Sqrt[((σD)^2)+(u^2)])],InverseCDF[NormalDistribution[μH,(Sqrt[((σH)^2)+(u^2)])],1-x]]

auc[μD_,μH_,σD_,σH_,u_]:=CDF[NormalDistribution[0,1],((μD-μH)/(Sqrt[((σD)^2)+((σH)^2)+2(u^2)]))]

Manipulate[
Plot[{Tooltip[roc[x,μD,μH,σD,σH,u1],"ROC curve of 1st test"],Tooltip[roc[ x,μD,μH,σD,σH,u2],"ROC curve of 2nd test"]},{x,0,1},PlotRange->{{0,1},{0,1}},AspectRatio->1,AxesLabel->{"1-specificity","sensitivity"},PlotLabel->Row[{"ratio of the two areas = ",N[auc[μD,μH,σD,σH,u1]/auc[μD,μH,σD,σH,u2],4]}],AxesOrigin->{0,1},PlotStyle->{ColorData["HTML","SlateBlue"], RGBColor[1,.47,0]} ,Filling->Bottom,ImageSize->{375,375}],
Style["nondiseased population",Bold],
{{μH,0,"mean"},0,6,.01,ImageSize->Tiny,Appearance->"Labeled"},
{{σH,1,"standard deviation"},0.1,6,.01,ImageSize->Tiny,Appearance->"Labeled"},
Style["diseased population",Bold],
{{μD,2,"mean"},0,6,.01,ImageSize->Tiny,Appearance->"Labeled"},
{{σD,1,"standard deviation"},0.1,6,.01,ImageSize->Tiny,Appearance->"Labeled"},
Style["uncertainty of measurement",Bold],
{{u1,0.2,"first test"},0,6,.01,ImageSize->Tiny,Appearance->"Labeled"},
{{u2,0.6,"second test"},0,6,.01,ImageSize->Tiny,Appearance->"Labeled"},SaveDefinitions->True,AutorunSequencing->{1,2,5,6},SynchronousUpdating->False]

Description

This Demonstration compares two receiver operating characteristic (ROC) plots of two diagnostic tests (first test: blue plot, second test: orange plot) measuring the same measurand. The comparisons are for normally distributed nondiseased and diseased populations, for various values of the mean and standard deviation of the populations, and of the uncertainty of measurement of the tests. A normal distribution of the uncertainty is assumed. The ratio of the areas under the ROC curves of the two diagnostic tests is calculated. The six parameters that you can vary using the sliders are measured in arbitrary units.

Snapshots

Details

The ROC plots are used in the evaluation of the clinical accuracy of a diagnostic test applied to a diseased and a nondiseased population. They display the sensitivity of the test versus 1-specificity. Sensitivity is the fraction of the diseased population with a positive test, while specificity is the fraction of the nondiseased population with a negative test. Therefore, the ROC plots display the true positive fraction versus the false positive fraction. Furthermore, the area under a ROC curve is used as an index of the diagnostic accuracy of the respective test.

This Demonstration could be useful in exploring ROC curves and evaluating the maximum medically permissible uncertainty of measurement of a diagnostic test. For example, in  the first snapshot the population data describes a bimodal distribution of serum glucose measurements with a non-diabetic and a diabetic population (Lim et al. 2002).

Reference

[1] T.-O. Lim, R. Bakri, Z. Morad, and M. A. Hamid, "Bimodality in Blood Glucose Distribution: Is It Universal?," Diabetes Care, 25(12), 2002 pp. 2212-2217.

Keywords

receiver operating characteristic curves
ROC curves
uncertainty of measurement
sensitivity
specificity
diagnostic test
clinical accuracy
diagnostic accuracy
permissible uncertainty
normal distribution
area under ROC curves

Disclaimer of Warranties

The notebook is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or Hellenic Complex Systems Laboratory or any of its members be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the notebook or the use or other dealings in the notebook.

About

Receiver Operating Characteristic Curves and Uncertainty of Measurement
Version Number 1.1.0
Hellenic Complex Systems Laboratory 2021
First published at Wolfram Demonstrations Project at: (HyperlinkURL["demonstrations.wolfram.com/ReceiverOperatingCharacteristicCurvesAndUncertaintyOfMeasure/", https://demonstrations.wolfram.com/ReceiverOperatingCharacteristicCurvesAndUncertaintyOfMeasure/])
The notebook is distributed under  an Attribution - NonCommercial - ShareAlike 4.0 International Creative Commons License.