SOCR logoBPAD

Shortcut Learning Detector

Compare a clinical model, a metadata-only model, and a within-site label-shuffle control to detect whether a classifier is exploiting acquisition shortcuts.

  • shortcut learning
  • confounding
  • negative control
  • site effects
Clinical model AUC
0.733
Metadata-only AUC
0.687
Shuffled-label AUC
0.687

Metadata-only predictions by site

chance00.10.20.30.40.50.60.70.80.91051015202530Predicted probability (metadata only)Count
Site ASite B
Verdict
Metadata-only beats chance, and so does the shuffled-label control: the model is overfitting to site noise, not detecting a real shortcut signal.