  Robotic systems are more present in our society everyday. In human-robot
environments, it is crucial that end-users may correctly understand their
robotic team-partners, in order to collaboratively complete a task. To increase
action understanding, users demand more explainability about the decisions by
the robot in particular situations. Recently, explainable robotic systems have
emerged as an alternative focused not only on completing a task satisfactorily,
but also on justifying, in a human-like manner, the reasons that lead to making
a decision. In reinforcement learning scenarios, a great effort has been
focused on providing explanations using data-driven approaches, particularly
from the visual input modality in deep learning-based systems. In this work, we
focus rather on the decision-making process of reinforcement learning agents
performing a task in a robotic scenario. Experimental results are obtained
using 3 different set-ups, namely, a deterministic navigation task, a
stochastic navigation task, and a continuous visual-based sorting object task.
As a way to explain the goal-driven robot's actions, we use the probability of
success computed by three different proposed approaches: memory-based,
learning-based, and introspection-based. The difference between these
approaches is the amount of memory required to compute or estimate the
probability of success as well as the kind of reinforcement learning
representation where they could be used. In this regard, we use the
memory-based approach as a baseline since it is obtained directly from the
agent's observations. When comparing the learning-based and the
introspection-based approaches to this baseline, both are found to be suitable
alternatives to compute the probability of success, obtaining high levels of
similarity when compared using both the Pearson's correlation and the mean
squared error.
