  Discovering the underlying dynamics of complex systems from data is an
important practical topic. Constrained optimization algorithms are widely
utilized and lead to many successes. Yet, such purely data-driven methods may
bring about incorrect physics in the presence of random noise and cannot easily
handle the situation with incomplete data. In this paper, a new iterative
learning algorithm for complex turbulent systems with partial observations is
developed that alternates between identifying model structures, recovering
unobserved variables, and estimating parameters. First, a causality-based
learning approach is utilized for the sparse identification of model
structures, which takes into account certain physics knowledge that is
pre-learned from data. It has unique advantages in coping with indirect
coupling between features and is robust to the stochastic noise. A practical
algorithm is designed to facilitate the causal inference for high-dimensional
systems. Next, a systematic nonlinear stochastic parameterization is built to
characterize the time evolution of the unobserved variables. Closed analytic
formula via an efficient nonlinear data assimilation is exploited to sample the
trajectories of the unobserved variables, which are then treated as synthetic
observations to advance a rapid parameter estimation. Furthermore, the
localization of the state variable dependence and the physics constraints are
incorporated into the learning procedure, which mitigate the curse of
dimensionality and prevent the finite time blow-up issue. Numerical experiments
show that the new algorithm succeeds in identifying the model structure and
providing suitable stochastic parameterizations for many complex nonlinear
systems with chaotic dynamics, spatiotemporal multiscale structures,
intermittency, and extreme events.
