  We present a neural network-based system for long-term, multi-action human
motion synthesis. The system, dubbed as NEURAL MARIONETTE, can produce
high-quality and meaningful motions with smooth transitions from simple user
input, including a sequence of action tags with expected action duration, and
optionally a hand-drawn moving trajectory if the user specifies. The core of
our system is a novel Transformer-based motion generation model, namely
MARIONET, which can generate diverse motions given action tags. Different from
existing motion generation models, MARIONET utilizes contextual information
from the past motion clip and future action tag, dedicated to generating
actions that can smoothly blend historical and future actions. Specifically,
MARIONET first encodes target action tag and contextual information into an
action-level latent code. The code is unfolded into frame-level control signals
via a time unrolling module, which could be then combined with other
frame-level control signals like the target trajectory. Motion frames are then
generated in an auto-regressive way. By sequentially applying MARIONET, the
system NEURAL MARIONETTE can robustly generate long-term, multi-action motions
with the help of two simple schemes, namely "Shadow Start" and "Action
Revision". Along with the novel system, we also present a new dataset dedicated
to the multi-action motion synthesis task, which contains both action tags and
their contextual information. Extensive experiments are conducted to study the
action accuracy, naturalism, and transition smoothness of the motions generated
by our system.
