  Scientific Machine Learning (SciML) is a burgeoning field that
synergistically combines domain-aware and interpretable models with agnostic
machine learning techniques. In this work, we introduce GOKU-UI, an evolution
of the SciML generative model GOKU-nets. GOKU-UI not only broadens the original
model's spectrum to incorporate other classes of differential equations, such
as Stochastic Differential Equations (SDEs), but also integrates attention
mechanisms and a novel multiple shooting training strategy in the latent space.
These modifications have led to a significant increase in its performance in
both reconstruction and forecast tasks, as demonstrated by our evaluation of
simulated and empirical data. Specifically, GOKU-UI outperformed all baseline
models on synthetic datasets even with a training set 16-fold smaller,
underscoring its remarkable data efficiency. Furthermore, when applied to
empirical human brain data, while incorporating stochastic Stuart-Landau
oscillators into its dynamical core, our proposed enhancements markedly
increased the model's effectiveness in capturing complex brain dynamics. This
augmented version not only surpassed all baseline methods in the reconstruction
task, but also demonstrated lower prediction error of future brain activity up
to 15 seconds ahead. By training GOKU-UI on resting state fMRI data, we encoded
whole-brain dynamics into a latent representation, learning a low-dimensional
dynamical system model that could offer insights into brain functionality and
open avenues for practical applications such as the classification of mental
states or psychiatric conditions. Ultimately, our research provides further
impetus for the field of Scientific Machine Learning, showcasing the potential
for advancements when established scientific insights are interwoven with
modern machine learning.
