  Reinforcement learning (RL) is a popular technique that allows an agent to
learn by trial and error while interacting with a dynamic environment. The
traditional Reinforcement Learning (RL) approach has been successful in
learning and predicting Euclidean robotic manipulation skills such as
positions, velocities, and forces. However, in robotics, it is common to
encounter non-Euclidean data such as orientation or stiffness, and failing to
account for their geometric nature can negatively impact learning accuracy and
performance. In this paper, to address this challenge, we propose a novel
framework for RL that leverages Riemannian geometry, which we call Geometric
Reinforcement Learning (G-RL), to enable agents to learn robotic manipulation
skills with non-Euclidean data. Specifically, G-RL utilizes the tangent space
in two ways: a tangent space for parameterization and a local tangent space for
mapping to a nonEuclidean manifold. The policy is learned in the
parameterization tangent space, which remains constant throughout the training.
The policy is then transferred to the local tangent space via parallel
transport and projected onto the non-Euclidean manifold. The local tangent
space changes over time to remain within the neighborhood of the current
manifold point, reducing the approximation error. Therefore, by introducing a
geometrically grounded pre- and post-processing step into the traditional RL
pipeline, our G-RL framework enables several model-free algorithms designed for
Euclidean space to learn from non-Euclidean data without modifications.
Experimental results, obtained both in simulation and on a real robot, support
our hypothesis that G-RL is more accurate and converges to a better solution
than approximating non-Euclidean data.
