  We consider a new problem of adapting a human mesh reconstruction model to
out-of-domain streaming videos, where performance of existing SMPL-based models
are significantly affected by the distribution shift represented by different
camera parameters, bone lengths, backgrounds, and occlusions. We tackle this
problem through online adaptation, gradually correcting the model bias during
testing. There are two main challenges: First, the lack of 3D annotations
increases the training difficulty and results in 3D ambiguities. Second,
non-stationary data distribution makes it difficult to strike a balance between
fitting regular frames and hard samples with severe occlusions or dramatic
changes. To this end, we propose the Dynamic Bilevel Online Adaptation
algorithm (DynaBOA). It first introduces the temporal constraints to compensate
for the unavailable 3D annotations, and leverages a bilevel optimization
procedure to address the conflicts between multi-objectives. DynaBOA provides
additional 3D guidance by co-training with similar source examples retrieved
efficiently despite the distribution shift. Furthermore, it can adaptively
adjust the number of optimization steps on individual frames to fully fit hard
samples and avoid overfitting regular frames. DynaBOA achieves state-of-the-art
results on three out-of-domain human mesh reconstruction benchmarks.
