  Human video motion transfer has a wide range of applications in multimedia,
computer vision and graphics. Recently, due to the rapid development of
Generative Adversarial Networks (GANs), there has been significant progress in
the field. However, almost all existing GAN-based works are prone to address
the mapping from human motions to video scenes, with scene appearances are
encoded individually in the trained models. Therefore, each trained model can
only generate videos with a specific scene appearance, new models are required
to be trained to generate new appearances. Besides, existing works lack the
capability of appearance control. For example, users have to provide video
records of wearing new clothes or performing in new backgrounds to enable
clothes or background changing in their synthetic videos, which greatly limits
the application flexibility. In this paper, we propose GAC-GAN, a general
method for appearance-controllable human video motion transfer. To enable
general-purpose appearance synthesis, we propose to include appearance
information in the conditioning inputs. Thus, once trained, our model can
generate new appearances by altering the input appearance information. To
achieve appearance control, we first obtain the appearance-controllable
conditioning inputs and then utilize a two-stage GAC-GAN to generate the
corresponding appearance-controllable outputs, where we utilize an ACGAN loss
and a shadow extraction module for output foreground and background appearance
control respectively. We further build a solo dance dataset containing a large
number of dance videos for training and evaluation. Experimental results show
that, our proposed GAC-GAN can not only support appearance-controllable human
video motion transfer but also achieve higher video quality than state-of-art
methods.
