Predictive values from sensitivity, specificity and prevalence (with Mercaldo CIs and natural frequencies)

https://udgca1190.com.mx/en/herramientas/bioestadistica/valores-predictivos

Inputs

Sensitivity (Sn)

From the validation study or the literature; from 0 to 1 or as a percentage (85 = 85%).

Specificity (Sp)

From 0 to 1 or as a percentage.

Prevalence where the test will be used

That of the population where the test will be applied, or the patient's pre-test probability; not that of the validation study.

Diseased in the validation study (optional)

People with the disease in the study from which Sn and Sp come. With both sizes the CI is computed (Mercaldo 2007); leave it empty or at 0 if unknown.

Non-diseased in the validation study (optional)

People without the disease in that same study.

Confidence level

Example loaded

Illustrative example: rapid NS1 antigen test with 85% sensitivity and 95% specificity (estimated in 80 diseased and 120 non-diseased people), applied where the prevalence of dengue among febrile patients is 30% (fictitious data).

Illustrative data, not real.

Results

Positive predictive value (PPV)

87.9%

76.9% to 94.1%

95% CI · Mercaldo logit

Negative predictive value (NPV)

93.7%

89.8% to 96.1%

95% CI · Mercaldo logit

LR+

17.00

LR−

0.16

True positives per 1,000

255

out of every 1,000 people tested

False positives per 1,000

out of every 1,000 people tested

False negatives per 1,000

out of every 1,000 people tested

True negatives per 1,000

665

out of every 1,000 people tested

Interpretation

If the test (Sn 85.0%, Sp 95.0%) is applied to 1,000 people with a prevalence of 30.0%, 255 true positives and 35 false positives are expected: 87.9% of the positive results will belong to people with the disease (PPV). There will be 45 false negatives and 665 true negatives: 93.7% of the negative results will belong to people without the disease (NPV).

With 80 diseased and 120 non-diseased people in the validation study, the 95% CI of the PPV is 76.9% to 94.1% and that of the NPV is 89.8% to 96.1% (Mercaldo 2007 logit method, with prevalence held fixed).

LR+ = 17.00 and LR− = 0.16: these two figures do not depend on prevalence and are the ones carried over to any pre-test probability with the "Post-test probability" calculator.

The prevalence must be that of the population where the test will be used (or the patient's pre-test probability), not that of the validation study: with the same Sn and Sp, PPV falls when the disease is rare and NPV falls when it is common (Vecchio 1966). The chart shows both values for any prevalence.

PPV and NPV against prevalence, with the entered prevalence marked

Explanation

Sensitivity and specificity describe the test, but the clinician's question is different: if the result is positive, how likely is it that the person has the disease? That is the positive predictive value (PPV); the negative one (NPV) is the probability of being disease-free given a negative result. Both depend on prevalence (Vecchio 1966): with the same test, PPV falls when the disease is rare and NPV falls when it is common.

That is why the predictive values of the validation study do not carry over to the clinic: they must be recomputed with the prevalence of the population where the test will be used (or with the patient's pre-test probability). This calculator does that with Bayes' theorem.

Natural frequencies (Gigerenzer and Edwards 2003) explain it without formulas: out of every 1,000 people tested, how many have the disease and test positive, how many test positive without it, and likewise for the negatives. The PPV is simply the fraction of positives who really have the disease.

If the sizes of the validation study are known (diseased and non-diseased people from whom Sn and Sp were estimated), the confidence interval of each predictive value is computed with the logit method of Mercaldo, Lau and Zhou (2007), which propagates the uncertainty of Sn and Sp while holding prevalence fixed. When Sn or Sp equal 0 or 1 its adjusted variant is used (0.5 added to each cell).

Equations

\mathrm{PPV}=\frac{\Sn\cdot P}{\Sn\cdot P+(1-\Sp)(1-P)},\qquad \mathrm{NPV}=\frac{\Sp\,(1-P)}{\Sp\,(1-P)+(1-\Sn)\,P}

$\Sn,\ \Sp$: sensitivity and specificity
$P$: prevalence (pre-test probability) where the test will be used

Bayes' theorem (1763) applied to a positive and to a negative result.

\se[\logit \mathrm{PPV}]=\sqrt{\frac{1-\Sn}{\Sn\,n_D}+\frac{\Sp}{(1-\Sp)\,n_{\bar D}}},\qquad \se[\logit \mathrm{NPV}]=\sqrt{\frac{1-\Sp}{\Sp\,n_{\bar D}}+\frac{\Sn}{(1-\Sn)\,n_D}}

$n_D,\ n_{\bar D}$: diseased and non-diseased in the validation study
$\logit p=\ln\frac{p}{1-p}$: logit scale; the interval is built on that scale and mapped back with expit(x) = 1/(1 + e^{−x})

Logit interval of Mercaldo, Lau and Zhou (2007): CI = expit(logit PPV ± z·SE). Adjusted variant: Sn and Sp are replaced by (Sn·n_D + 0.5)/(n_D + 1) and (Sp·n_D̄ + 0.5)/(n_D̄ + 1), with n + 1 in each group.

\mathrm{TP}=1000\,P\,\Sn,\qquad \mathrm{FN}=1000\,P\,(1-\Sn),\qquad \mathrm{FP}=1000\,(1-P)(1-\Sp),\qquad \mathrm{TN}=1000\,(1-P)\,\Sp

Natural frequencies per 1,000 people tested (Gigerenzer and Edwards 2003); PPV = TP/(TP + FP) and NPV = TN/(TN + FN).

R code

# Predictive values from sensitivity, specificity and prevalence - Bioestadistica abierta, UDG-CA-1190
# Runs as is in R, RStudio or webR; prints the results as JSON at the end.
library(jsonlite)

sn <- 0.85; sp <- 0.95; prev <- 0.3   # sensitivity, specificity and prevalence where the test will be used (0-1)
n_d <- 80; n_nd <- 120             # diseased and non-diseased in the validation study; 0 = unknown (no CI)
nivel <- 0.95
z <- qnorm(1 - (1 - nivel) / 2)

vpp_punto <- sn * prev / (sn * prev + (1 - sp) * (1 - prev))         # Bayes' theorem
vpn_punto <- sp * (1 - prev) / (sp * (1 - prev) + (1 - sn) * prev)
lr_pos <- sn / (1 - sp)
lr_neg <- (1 - sn) / sp

# Logit confidence interval of Mercaldo, Lau & Zhou (2007); with Sn or Sp at 0 or 1 the standard
# error is undefined and the adjusted logit is used (0.5 added to each cell of the validation study)
logit <- function(p) log(p / (1 - p))
expit <- function(x) 1 / (1 + exp(-x))
ic_logit <- function(s, e, d, nd) {      # s = Sn, e = Sp, d = diseased, nd = non-diseased
  vpp <- s * prev / (s * prev + (1 - e) * (1 - prev))
  vpn <- e * (1 - prev) / (e * (1 - prev) + (1 - s) * prev)
  ee_vpp <- sqrt((1 - s) / (s * d) + e / ((1 - e) * nd))
  ee_vpn <- sqrt((1 - e) / (e * nd) + s / ((1 - s) * d))
  list(vpp = c(expit(logit(vpp) - z * ee_vpp), expit(logit(vpp) + z * ee_vpp)),
       vpn = c(expit(logit(vpn) - z * ee_vpn), expit(logit(vpn) + z * ee_vpn)))
}
if (n_d > 0 && n_nd > 0) {
  ajustado <- sn %in% c(0, 1) || sp %in% c(0, 1)
  ic <- if (ajustado) ic_logit((sn * n_d + 0.5) / (n_d + 1), (sp * n_nd + 0.5) / (n_nd + 1), n_d + 1, n_nd + 1) else ic_logit(sn, sp, n_d, n_nd)
  vpp <- c(vpp_punto, ic$vpp); vpn <- c(vpn_punto, ic$vpn)
} else {
  vpp <- vpp_punto; vpn <- vpn_punto     # without the study sizes there is no interval
}

# Natural frequencies per 1,000 people tested (Gigerenzer & Edwards 2003)
vp_mil <- 1000 * prev * sn;       fn_mil <- 1000 * prev * (1 - sn)
fp_mil <- 1000 * (1 - prev) * (1 - sp); vn_mil <- 1000 * (1 - prev) * sp

res <- list(vpp = vpp, vpn = vpn, lr_pos = lr_pos, lr_neg = lr_neg,
            vp_mil = vp_mil, fp_mil = fp_mil, fn_mil = fn_mil, vn_mil = vn_mil)
cat(toJSON(res, auto_unbox = TRUE, digits = NA))

This is the very code that validates the calculator: copy it and run it in R or RStudio to reproduce the result.

In-browser verification with R will arrive in a forthcoming version; meanwhile, copy the code and run it in RStudio.

Methods for a manuscript

Predictive values were derived from sensitivity, specificity and the assumed prevalence with Bayes' theorem [4,1]. The 95% confidence intervals were computed with the logit method of Mercaldo et al. [2] from the sizes of the validation study (80 with the disease and 120 without it), with prevalence held fixed. Results were also expressed as natural frequencies per 1,000 people [3]. Calculations used the "Predictive values" calculator of Bioestadística abierta (Research Group UDG-CA-1190, https://udgca1190.com.mx/en/herramientas/bioestadistica/valores-predictivos), verified against R.

A paragraph ready for the Methods section; bracketed numbers refer to the reference list.

References

01 Vecchio TJ. Predictive value of a single diagnostic test in unselected populations. New England Journal of Medicine. 1966;274(21):1171–1173. doi:10.1056/NEJM196605262742104 PMID: 5934954 Original source
02 Mercaldo ND, Lau KF, Zhou XH. Confidence intervals for predictive values with an emphasis to case-control studies. Statistics in Medicine. 2007;26(10):2170–2183. doi:10.1002/sim.2677 PMID: 16927452 Original source
03 Gigerenzer G, Edwards A. Simple tools for understanding risks: from innumeracy to insight. BMJ. 2003;327(7417):741–744. doi:10.1136/bmj.327.7417.741 PMID: 14512488 Original source
04 Bayes T. An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London. 1763;53:370–418. doi:10.1098/rstl.1763.0053 Original source
05 Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ. 1994;309(6947):102. doi:10.1136/bmj.309.6947.102 PMID: 8038641 Didactic reading
06 Fletcher RH, Fletcher SW, Fletcher GS. Clinical Epidemiology: The Essentials. 5th ed. Philadelphia: Lippincott Williams & Wilkins; 2014. Didactic reading