  Data-driven models for nonlinear dynamical systems based on approximating the
underlying Koopman operator or generator have proven to be successful tools for
forecasting, feature learning, state estimation, and control. It has become
well known that the Koopman generators for control-affine systems also have
affine dependence on the input, leading to convenient finite-dimensional
bilinear approximations of the dynamics. Yet there are still two main obstacles
that limit the scope of current approaches for approximating the Koopman
generators of systems with actuation. First, the performance of existing
methods depends heavily on the choice of basis functions over which the Koopman
generator is to be approximated; and there is currently no universal way to
choose them for systems that are not measure preserving. Secondly, if we do not
observe the full state, then it becomes necessary to account for the dependence
of the output time series on the sequence of supplied inputs when constructing
observables to approximate Koopman operators. To address these issues, we write
the dynamics of observables governed by the Koopman generator as a bilinear
hidden Markov model, and determine the model parameters using the
expectation-maximization (EM) algorithm. The E-step involves a standard Kalman
filter and smoother, while the M-step resembles control-affine dynamic mode
decomposition for the generator. We demonstrate the performance of this method
on three examples, including recovery of a finite-dimensional Koopman-invariant
subspace for an actuated system with a slow manifold; estimation of Koopman
eigenfunctions for the unforced Duffing equation; and model-predictive control
of a fluidic pinball system based only on noisy observations of lift and drag.
