  Learning controllers that reproduce legged locomotion in nature has been a
long-time goal in robotics and computer graphics. While yielding promising
results, recent approaches are not yet flexible enough to be applicable to
legged systems of different morphologies. This is partly because they often
rely on precise motion capture references or elaborate learning environments
that ensure the naturality of the emergent locomotion gaits but prevent
generalization. This work proposes a generic approach for ensuring realism in
locomotion by guiding the learning process with the spring-loaded inverted
pendulum model as a reference. Leveraging on the exploration capacities of
Reinforcement Learning (RL), we learn a control policy that fills in the
information gap between the template model and full-body dynamics required to
maintain stable and periodic locomotion. The proposed approach can be applied
to robots of different sizes and morphologies and adapted to any RL technique
and control architecture. We present experimental results showing that even in
a model-free setup and with a simple reactive control architecture, the learned
policies can generate realistic and energy-efficient locomotion gaits for a
bipedal and a quadrupedal robot. And most importantly, this is achieved without
using motion capture, strong constraints in the dynamics or kinematics of the
robot, nor prescribing limb coordination. We provide supplemental videos for
qualitative analysis of the naturality of the learned gaits.
