  Learning effective continuous control policies in high-dimensional systems,
including musculoskeletal agents, remains a significant challenge. Over the
course of biological evolution, organisms have developed robust mechanisms for
overcoming this complexity to learn highly sophisticated strategies for motor
control. What accounts for this robust behavioral flexibility? Modular control
via muscle synergies, i.e. coordinated muscle co-contractions, is considered to
be one putative mechanism that enables organisms to learn muscle control in a
simplified and generalizable action space. Drawing inspiration from this
evolved motor control strategy, we use physiologically accurate human hand and
leg models as a testbed for determining the extent to which a Synergistic
Action Representation (SAR) acquired from simpler tasks facilitates learning
more complex tasks. We find in both cases that SAR-exploiting policies
significantly outperform end-to-end reinforcement learning. Policies trained
with SAR were able to achieve robust locomotion on a wide set of terrains with
high sample efficiency, while baseline approaches failed to learn meaningful
behaviors. Additionally, policies trained with SAR on a multiobject
manipulation task significantly outperformed (>70% success) baseline approaches
(<20% success). Both of these SAR-exploiting policies were also found to
generalize zero-shot to out-of-domain environmental conditions, while policies
that did not adopt SAR failed to generalize. Finally, we establish the
generality of SAR on broader high-dimensional control problems using a robotic
manipulation task set and a full-body humanoid locomotion task. To the best of
our knowledge, this investigation is the first of its kind to present an
end-to-end pipeline for discovering synergies and using this representation to
learn high-dimensional continuous control across a wide diversity of tasks.
