Cost-Sensitive Threshold Explorer

Price a missed malignancy against an unnecessary operation and read the decision threshold straight off the cost ratio.

decision theory
threshold
cost
ROC

ROC — Out-of-fold (train)

Total expected cost vs. threshold

Confusion matrix at the cost-optimal threshold

	Pred +	Pred −
True +	136	5
True −	76	183

Operating metrics

Metric	Value
Threshold	0.333
Sensitivity	0.965
Specificity	0.707
PPV	0.642
NPV	0.973
Total cost	430
Cost if everyone treated	1295
Cost if nobody treated	1410

The threshold is a value judgement, not a statistic

With C_FP = 5 and C_FN = 10, the cost-minimizing cut-point is C_FP/(C_FP+C_FN) = 0.333. Nothing about the model changes when you move the slider — only the price you attach to the two kinds of mistake. Note also how the external cohort, with lower discrimination and lower prevalence, pays a higher total cost at the same threshold.

Preparing the simulation…

Conceptual bridges & related modules

QRS Detection & HRV Analyzer
downstream classifier · Chapter 7
Compare error costs in detection and downstream decision thresholds.