Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Measuring the Reliability of Reinforcement Learning Algorithms   Lack of reliability is a well-known issue for reinforcement learning (RL)
algorithms. This problem has gained increasing attention in recent years, and
efforts to improve it have grown substantially. To aid RL researchers and
production users with the evaluation and improvement of reliability, we propose
a set of metrics that quantitatively measure different aspects of reliability.
In this work, we focus on variability and risk, both during training and after
learning (on a fixed policy). We designed these metrics to be general-purpose,
and we also designed complementary statistical tests to enable rigorous
comparisons on these metrics. In this paper, we first describe the desired
properties of the metrics and their design, the aspects of reliability that
they measure, and their applicability to different scenarios. We then describe
the statistical tests and make additional practical recommendations for
reporting results. The metrics and accompanying statistical tools have been
made available as an open-source library at
https://github.com/google-research/rl-reliability-metrics. We apply our metrics
to a set of common RL algorithms and environments, compare them, and analyze
the results.

31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
0
The categories of this paper are: hep-ph
24
The categories of this paper are: dg-ga
10
The categories of this paper are: q-bio
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
6
The categories of this paper are: astro-ph
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat