  Purpose. NTCP modelling is rapidly embracing DL methods as the need to
include spatial dose information is acknowledged. Finding the most appropriate
way of combining radiation dose distribution images and clinical data involves
technical challenges and requires domain knowledge. We propose different data
fusion strategies that we hope will serve as a starting point for future DL
NTCP studies. Methods. Early, joint and late DL multi-modality fusion
strategies were compared using clinical variables and mandibular radiation dose
distribution volumes. The discriminative performance of the multi-modality
models was compared to that of single-modality models. All the experiments were
conducted on a control-case matched cohort of 92 ORN cases and 92 controls from
a single institution. Results. The highest ROC AUC score was obtained with the
late fusion model (0.70), but no statistically significant differences in
discrimination performance were observed between strategies. While late fusion
was the least technically complex strategy, its design did not model the
inter-modality interactions that are required for NTCP modelling. Joint fusion
involved the most complex design but resulted in a single network training
process which included intra- and inter-modality interactions in its model
parameter optimisation. Conclusions. This is the first study that compares
different strategies for including image data into DL NTCP models in
combination with lower dimensional data such as clinical variables. The
discrimination performance of such multi-modality NTCP models and the choice of
fusion strategy will depend on the distribution and quality of both types of
data. We encourage future DL NTCP studies to report on different fusion
strategies to better justify their choice of DL pipeline.
