Multimodal Fusion: Incremental Value Explorer
Fuse a clinical model and an imaging model by weighted averaging, and test whether the combination actually beats the stronger baseline.
- multimodal
- fusion
- incremental value
- ROC
ROC curves: clinical vs. imaging vs. fusion
Clinical only (AUC=0.95)Imaging only (AUC=0.97)Fusion (AUC=0.99)
Metrics table
| Model | AUC | Brier |
|---|---|---|
| Clinical only | 0.952 | 0.114 |
| Imaging only | 0.968 | 0.095 |
| Fusion | 0.994 | 0.091 |
Clinical vs. imaging predicted probabilities
Agreement is low; fusion only helps if errors are uncorrelated.
Incremental value depends on independent errors
Fusion beats both single-modality models only when the clinical and imaging scores make largely uncorrelated mistakes — that is what the scatter plot's spread away from the diagonal represents. On the internal set a modest imaging weight can lift AUC above either baseline. Switch to the external set: the imaging model typically degrades more than the clinical model under acquisition shift, so the same fusion weight can now hurt, and the optimal weight shifts back toward the clinical-only baseline. Incremental value should always be judged relative to the strongest single-modality model, not against no model at all.