  We study a challenging task, conditional human motion generation, which
produces plausible human motion sequences according to various conditional
inputs, such as action classes or textual descriptors. Since human motions are
highly diverse and have a property of quite different distribution from
conditional modalities, such as textual descriptors in natural languages, it is
hard to learn a probabilistic mapping from the desired conditional modality to
the human motion sequences. Besides, the raw motion data from the motion
capture system might be redundant in sequences and contain noises; directly
modeling the joint distribution over the raw motion sequences and conditional
modalities would need a heavy computational overhead and might result in
artifacts introduced by the captured noises. To learn a better representation
of the various human motion sequences, we first design a powerful Variational
AutoEncoder (VAE) and arrive at a representative and low-dimensional latent
code for a human motion sequence. Then, instead of using a diffusion model to
establish the connections between the raw motion sequences and the conditional
inputs, we perform a diffusion process on the motion latent space. Our proposed
Motion Latent-based Diffusion model (MLD) could produce vivid motion sequences
conforming to the given conditional inputs and substantially reduce the
computational overhead in both the training and inference stages. Extensive
experiments on various human motion generation tasks demonstrate that our MLD
achieves significant improvements over the state-of-the-art methods among
extensive human motion generation tasks, with two orders of magnitude faster
than previous diffusion models on raw motion sequences.
