SOCR logoBPAD

Acquisition Shift Explorer

Change slice thickness, reconstruction kernel and noise, then watch a model trained on one protocol degrade on another.

  • domain shift
  • external validation
  • protocol
  • generalization
Internal test AUC
0.660
External AUC
0.697
Generalization gap
-0.037
modest

ROC: same model, two acquisition protocols

00.10.20.30.40.50.60.70.80.9100.20.40.60.811 − specificitySensitivity
Internal (thin slice)External protocol

Texture feature distribution by site

-4-3-2-10123400.10.20.30.40.5GLCM contrast (arbitrary units)Density
Development siteExternal site
Reading the gap
With concept shift at zero the entire loss is covariate shift: the tumours are unchanged, only their measurements moved. Harmonization recovers much of that. Any gap that persists once harmonization is on, and all of the loss once concept shift is non-zero, cannot be fixed by rescaling features.