Calculation of Diagnostic Accuracy Measures

Theodora Chatzimichail, MD  
tc@hcsl.com
Aristides T. Hatjimihail, MD, PhD
 ath@hcsl.com

Hellenic Complex Systems Laboratory, Kostis Palamas 21, 66131 Drama, Greece.

Initialization Code

Get["HypothesisTesting`"]

ci[p_,n_,cl_]:=Block[{z=InverseCDF[NormalDistribution[0,1],((1+cl)/2)],q=1-p},{If[p>0,((2n p+(z^2)-1-z(Sqrt[(z^2)-2-(1/n)+4p(n q+1)]))/(2(n+(z^2)))),0],If[p<1,((2n p+(z^2)+1+z(Sqrt[(z^2)+2-(1/n)+4p(n q-1)]))/(2(n+(z^2)))),1]}]
se[tn_,fp_,fn_,tp_]:=(tp/(fn+tp))

seci[tn_,fp_,fn_,tp_,cl_]:=ci[se[tn,fp,fn,tp],fn+tp,cl]

sp[tn_,fp_,fn_,tp_]:=(tn/(tn+fp))

spci[tn_,fp_,fn_,tp_,cl_]:=ci[sp[tn,fp,fn,tp],tn+fp,cl]

pr[tn_,fp_,fn_,tp_]:=((fn+tp)/(tn+fp+fn+tp))

ppv[tn_,fp_,fn_,tp_]:=(tp/(tp+fp))

ppvci[tn_,fp_,fn_,tp_,cl_]:=ci[ppv[tn,fp,fn,tp],tp+fp,cl]

npv[tn_,fp_,fn_,tp_]:=(tn/(tn+fn))

npvci[tn_,fp_,fn_,tp_,cl_]:=ci[npv[tn,fp,fn,tp],tn+fn,cl]

or[tn_,fp_,fn_,tp_]:=(((se[tn,fp,fn,tp])/(1-se[tn,fp,fn,tp]))/((1-sp[tn,fp,fn,tp])/(sp[tn,fp,fn,tp])))

logorse[tn_,fp_,fn_,tp_]:=(Sqrt[(1/tp)+(1/fp)+(1/tn)+(1/fn)])

orci[tn_,fp_,fn_,tp_,cl_]:=Exp[Log[or[tn,fp,fn,tp]]+logorse[tn,fp,fn,tp]NormalCI[0,1,ConfidenceLevel->cl]]

lrp[tn_,fp_,fn_,tp_]:=((se[tn,fp,fn,tp])/(1-sp[tn,fp,fn,tp]))

loglrpse[tn_,fp_,fn_,tp_]:=(Sqrt[(1/tp)+(1/(tp+fn))+(1/fp)+(1/(tn+fp))])

lrpci[tn_,fp_,fn_,tp_,cl_]:=Exp[Log[lrp[tn,fp,fn,tp]]+loglrpse[tn,fp,fn,tp]NormalCI[0,1,ConfidenceLevel->cl]]

lrn[tn_,fp_,fn_,tp_]:=((1-se[tn,fp,fn,tp])/(sp[tn,fp,fn,tp]))

loglrnse[tn_,fp_,fn_,tp_]:=(Sqrt[(1/fn)+(1/(tp+fn))+(1/tn)+(1/(tn+fp))])

lrnci[tn_,fp_,fn_,tp_,cl_]:=Exp[Log[lrn[tn,fp,fn,tp]]+loglrnse[tn,fp,fn,tp]NormalCI[0,1,ConfidenceLevel->cl]]

pop[tn_,fp_,fn_,tp_]:=tn+fp+fn+tp

ndis[tn_,fp_,fn_,tp_]:=tn+fp

dis[tn_,fp_,fn_,tp_]:=fn+tp

popstat[tn_,fp_,fn_,tp_]:=Text@Grid[Transpose[{{Style["population statistics",Bold],"total population","nondiseased","diseased","prevalence of disease"},{SpanFromLeft,pop[tn,fp,fn,tp],ndis[tn,fp,fn,tp],dis[tn,fp,fn,tp],NumberForm[N@pr[tn,fp,fn,tp],{8,6}]}}],Frame->All, FrameStyle->Blend[{Blue,Cyan},0.35],Spacings->{2,2},ItemSize->{{20,10},Automatic}]

damci[meas_,tn_,fp_,fn_,tp_,cl_]:=Text@Grid[Transpose[Switch[meas,se,{{Style[StringJoin["sensitivity ",ToString[100cl], "% confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@se[tn,fp,fn,tp],{8,6}],NumberForm[N@seci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@seci[tn,fp,fn,tp,cl][[2]],{8,6}]}},sp,{{Style[StringJoin["specificity ",ToString[100cl], "% confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@sp[tn,fp,fn,tp],{8,6}],NumberForm[N@spci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@spci[tn,fp,fn,tp,cl][[2]],{8,6}]}},ppv,{{Style[StringJoin["PPV ",ToString[100cl], " % confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@ppv[tn,fp,fn,tp],{8,6}],NumberForm[N@ppvci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@ppvci[tn,fp,fn,tp,cl][[2]],{8,6}]}},npv,{{Style[StringJoin["NPV ",ToString[100cl], "% confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@npv[tn,fp,fn,tp],{8,6}],NumberForm[N@npvci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@npvci[tn,fp,fn,tp,cl][[2]],{8,6}]}},or,{{Style[StringJoin["OR ",ToString[100cl], "% confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@or[tn,fp,fn,tp],{8,6}],NumberForm[N@orci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@orci[tn,fp,fn,tp,cl][[2]],{8,6}]}},lrp,{{Style[StringJoin["LR+ ",ToString[100cl], "% confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@lrp[tn,fp,fn,tp],{8,6}],NumberForm[N@lrpci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@lrpci[tn,fp,fn,tp,cl][[2]],{8,6}]}},lrn,{{Style[StringJoin["LR- ",ToString[100cl], "% confidence interval"],Bold],"point estimation","lower bound","upper bound"},{SpanFromLeft,NumberForm[N@lrn[tn,fp,fn,tp],{8,6}],NumberForm[N@lrnci[tn,fp,fn,tp,cl][[1]],{8,6}],NumberForm[N@lrnci[tn,fp,fn,tp,cl][[2]],{8,6}]}}]],Frame->All, FrameStyle->Orange,Spacings->{2,2},ItemSize->{{20,10},Automatic}]

Manipulate[
Grid[{{Pane[popstat[tn,fp,fn,tp], {420,200},Alignment->Center]},{Pane[damci[meas,tn,fp,fn,tp,cl], {420,200},Alignment->Center]}}],
Style["nondiseased population",Bold],
"negative test result",
{{tn,8550,""},1,100000,1,Appearance->"Labeled",ImageSize->Tiny},
"positive test result",
{{fp,950,""},1,100000,1,Appearance->"Labeled", ImageSize->Tiny} ,"",
Style["diseased population",Bold],
"positive test result",
{{tp,450,""},1,100000,1,Appearance->"Labeled",ImageSize->Tiny},
"negative test result",
{{fn,50,""},1,100000,1,Appearance->"Labeled", ImageSize->Tiny},"",
Style[Row[{Style["p",Italic],"-value"}],Bold],
{{cl,0.95,""},0.001,0.999,0.001,Appearance->"Labeled",ImageSize->Tiny},"",
Style["measure",Bold],
{{meas, se, ""},{se->"sensitivity",sp->"specificity", ppv->"positive predictive value", npv->"negative predictive value", or-> "odds ratio", lrp->"likelihood ratio for \na positive result", lrn->"likelihood ratio for \na negative result"}},
ControlPlacement->Left, ContinuousAction->False, SaveDefinitions->True]

Description

This Demonstration shows calculations of point estimations and confidence intervals for various accuracy measures of a diagnostic test for a disease. This is done for differing negative and positive test results of nondiseased and diseased populations and differing p-values for the estimations of the lower and upper bounds of the confidence intervals. The calculated measures are the sensitivity, the specificity, the positive predictive value (PPV), the negative predictive value (NPV), the (diagnostic) odds ratio (OR), the likelihood ratio for a positive test result (LR+) and the likelihood ratio for a negative test result (LR−). The measures can be selected using the menu. The negative and positive test results of the nondiseased and diseased populations, along with the p-value, are chosen using the sliders.

Snapshots

Details

Sensitivity refers to the fraction of the diseased population with a positive test result, while specificity is the fraction of the nondiseased population with a negative test result. Positive predictive value is the fraction of the population with a positive test result that is diseased, while negative predictive value is the fraction of the population with a negative test result that is not diseased. Prevalence of the disease is the ratio of the diseased population to the total (nondiseased and diseased) population. If we denote by sens the sensitivity, spec the specificity and pr the prevalence, we have:

OR=(sens/(1-sens))/((1-spec)/spec),

LR+ =(sens/(1-spec)),

LR- =((1-sens)/spec).

The Wilson score method with continuity correction is used [1] for calculating the confidence intervals of the sensitivity, the specificity, the positive predictive value and the negative predictive value.

For the calculation of the confidence intervals of the (diagnostic) odds ratio, the likelihood ratio for a positive test result and the likelihood ratio for a negative test result, it is assumed that their natural logarithms have asymptotically normal distributions [1].

This Demonstration is an extension of another Demonstration [2] and is appropriate as an educational and research tool.

References

[1] J. L. Fleiss, B. Levin and M. C. Paik, Statistical Methods for Rates and Proportions, 3rd ed., Hoboken, NJ: J. Wiley, 2003.

[2] T. Chatzimichaill, Calculator for Diagnostic Accuracy Measures. Wolfram Demonstrations Project, 2018. Available at: demonstrations.wolfram.com/CalculatorForDiagnosticAccuracyMeasures.

Keywords

sensitivity
specificity
diagnostic test
clinical accuracy
diagnostic accuracy
positive predictive value
negative predictive value
likelihood ratio
odds ratio
confidence intervals

Related Links

(HyperlinkDefault["Analysis of Diagnostic Accuracy Measures", demonstrations.wolfram.com/AnalysisOfDiagnosticAccuracyMeasures/, demonstrations.wolfram.com/AnalysisOfDiagnosticAccuracyMeasures/])

Disclaimer of Warranties

The notebook is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or Hellenic Complex Systems Laboratory or any of its members be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the notebook or the use or other dealings in the notebook.

About

Calculation of Diagnostic Accuracy Measures
Version Number 1.1.0
Hellenic Complex Systems Laboratory 2021
First published at Wolfram Demonstrations Project at: (HyperlinkURL["demonstrations.wolfram.com/CalculationOfDiagnosticAccuracyMeasures/", https://demonstrations.wolfram.com/CalculationOfDiagnosticAccuracyMeasures/])
The notebook is distributed under  an Attribution - NonCommercial - ShareAlike 4.0 International Creative Commons License.