  We present a universal motion representation that encompasses a comprehensive
range of motor skills for physics-based humanoid control. Due to the
high-dimensionality of humanoid control as well as the inherent difficulties in
reinforcement learning, prior methods have focused on learning skill embeddings
for a narrow range of movement styles (e.g. locomotion, game characters) from
specialized motion datasets. This limited scope hampers its applicability in
complex tasks. Our work closes this gap, significantly increasing the coverage
of motion representation space. To achieve this, we first learn a motion
imitator that can imitate all of human motion from a large, unstructured motion
dataset. We then create our motion representation by distilling skills directly
from the imitator. This is achieved using an encoder-decoder structure with a
variational information bottleneck. Additionally, we jointly learn a prior
conditioned on proprioception (humanoid's own pose and velocities) to improve
model expressiveness and sampling efficiency for downstream tasks. Sampling
from the prior, we can generate long, stable, and diverse human motions. Using
this latent space for hierarchical RL, we show that our policies solve tasks
using natural and realistic human behavior. We demonstrate the effectiveness of
our motion representation by solving generative tasks (e.g. strike, terrain
traversal) and motion tracking using VR controllers.
