  Large, open-source consortium datasets have spurred the development of new
and increasingly powerful machine learning approaches in brain connectomics.
However, one key question remains: are we capturing biologically relevant and
generalizable information about the brain, or are we simply overfitting to the
data? To answer this, we organized a scientific challenge, the Connectomics in
NeuroImaging Transfer Learning Challenge (CNI-TLC), held in conjunction with
MICCAI 2019. CNI-TLC included two classification tasks: (1) diagnosis of
Attention-Deficit/Hyperactivity Disorder (ADHD) within a pre-adolescent cohort;
and (2) transference of the ADHD model to a related cohort of Autism Spectrum
Disorder (ASD) patients with an ADHD comorbidity. In total, 240 resting-state
fMRI time series averaged according to three standard parcellation atlases,
along with clinical diagnosis, were released for training and validation (120
neurotypical controls and 120 ADHD). We also provided demographic information
of age, sex, IQ, and handedness. A second set of 100 subjects (50 neurotypical
controls, 25 ADHD, and 25 ASD with ADHD comorbidity) was used for testing.
Models were submitted in a standardized format as Docker images through ChRIS,
an open-source image analysis platform. Utilizing an inclusive approach, we
ranked the methods based on 16 different metrics. The final rank was calculated
using the rank product for each participant across all measures. Furthermore,
we assessed the calibration curves of each method. Five participants submitted
their model for evaluation, with one outperforming all other methods in both
ADHD and ASD classification. However, further improvements are needed to reach
the clinical translation of functional connectomics. We are keeping the CNI-TLC
open as a publicly available resource for developing and validating new
classification methodologies in the field of connectomics.
