  Data assimilation (DA) is a key component of many forecasting models in
science and engineering. DA allows one to estimate better initial conditions
using an imperfect dynamical model of the system and noisy/sparse observations
available from the system. Ensemble Kalman filter (EnKF) is a DA algorithm that
is widely used in applications involving high-dimensional nonlinear dynamical
systems. However, EnKF requires evolving large ensembles of forecasts using the
dynamical model of the system. This often becomes computationally intractable,
especially when the number of states of the system is very large, e.g., for
weather prediction. With small ensembles, the estimated background error
covariance matrix in the EnKF algorithm suffers from sampling error, leading to
an erroneous estimate of the analysis state (initial condition for the next
forecast cycle). In this work, we propose hybrid ensemble Kalman filter
(H-EnKF), which is applied to a two-layer quasi-geostrophic flow system as a
test case. This framework utilizes a pre-trained deep learning-based
data-driven surrogate that inexpensively generates and evolves a large
data-driven ensemble of the states of the system to accurately compute the
background error covariance matrix with less sampling error. The H-EnKF
framework estimates a better initial condition without the need for any ad-hoc
localization strategies. H-EnKF can be extended to any ensemble-based DA
algorithm, e.g., particle filters, which are currently difficult to use for
high dimensional systems.
