Cost-Sensitive Threshold Explorer
Price a missed malignancy against an unnecessary operation and read the decision threshold straight off the cost ratio.
- decision theory
- threshold
- cost
- ROC
ROC — Out-of-fold (train)
Total expected cost vs. threshold
Confusion matrix at the cost-optimal threshold
| Pred + | Pred − | |
|---|---|---|
| True + | 136 | 5 |
| True − | 76 | 183 |
Operating metrics
| Metric | Value |
|---|---|
| Threshold | 0.333 |
| Sensitivity | 0.965 |
| Specificity | 0.707 |
| PPV | 0.642 |
| NPV | 0.973 |
| Total cost | 430 |
| Cost if everyone treated | 1295 |
| Cost if nobody treated | 1410 |
The threshold is a value judgement, not a statistic
With C_FP = 5 and C_FN = 10, the cost-minimizing cut-point is C_FP/(C_FP+C_FN) = 0.333. Nothing about the model changes when you move the slider — only the price you attach to the two kinds of mistake. Note also how the external cohort, with lower discrimination and lower prevalence, pays a higher total cost at the same threshold.