  A central question in neuroscience is how self-organizing dynamic
interactions in the brain emerge on their relatively static structural
backbone. Due to the complexity of spatial and temporal dependencies between
different brain areas, fully comprehending the interplay between structure and
function is still challenging and an area of intense research. In this paper we
present a graph neural network (GNN) framework, to describe functional
interactions based on the structural anatomical layout. A GNN allows us to
process graph-structured spatio-temporal signals, providing a possibility to
combine structural information derived from diffusion tensor imaging (DTI) with
temporal neural activity profiles, like observed in functional magnetic
resonance imaging (fMRI). Moreover, dynamic interactions between different
brain regions learned by this data-driven approach can provide a multi-modal
measure of causal connectivity strength. We assess the proposed model's
accuracy by evaluating its capabilities to replicate empirically observed
neural activation profiles, and compare the performance to those of a vector
auto regression (VAR), like typically used in Granger causality. We show that
GNNs are able to capture long-term dependencies in data and also
computationally scale up to the analysis of large-scale networks. Finally we
confirm that features learned by a GNN can generalize across MRI scanner types
and acquisition protocols, by demonstrating that the performance on small
datasets can be improved by pre-training the GNN on data from an earlier and
different study. We conclude that the proposed multi-modal GNN framework can
provide a novel perspective on the structure-function relationship in the
brain. Therewith this approach can be promising for the characterization of the
information flow in brain networks.
