  We develop a class of interacting particle systems for implementing a maximum
marginal likelihood estimation (MMLE) procedure to estimate the parameters of a
latent variable model. We achieve this by formulating a continuous-time
interacting particle system which can be seen as a Langevin diffusion over an
extended state space of parameters and latent variables. In particular, we
prove that the parameter marginal of the stationary measure of this diffusion
has the form of a Gibbs measure where number of particles acts as the inverse
temperature parameter in classical settings for global optimisation. Using a
particular rescaling, we then prove geometric ergodicity of this system and
bound the discretisation error in a manner that is uniform in time and does not
increase with the number of particles. The discretisation results in an
algorithm, termed Interacting Particle Langevin Algorithm (IPLA) which can be
used for MMLE. We further prove nonasymptotic bounds for the optimisation error
of our estimator in terms of key parameters of the problem, and also extend
this result to the case of stochastic gradients covering practical scenarios.
We provide numerical experiments to illustrate the empirical behaviour of our
algorithm in the context of logistic regression with verifiable assumptions.
Our setting provides a straightforward way to implement a diffusion-based
optimisation routine compared to more classical approaches such as the
Expectation Maximisation (EM) algorithm, and allows for especially explicit
nonasymptotic bounds.
