Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Constraints Penalized Q-learning for Safe Offline Reinforcement Learning   We study the problem of safe offline reinforcement learning (RL), the goal is
to learn a policy that maximizes long-term reward while satisfying safety
constraints given only offline data, without further interaction with the
environment. This problem is more appealing for real world RL applications, in
which data collection is costly or dangerous. Enforcing constraint satisfaction
is non-trivial, especially in offline settings, as there is a potential large
discrepancy between the policy distribution and the data distribution, causing
errors in estimating the value of safety constraints. We show that na\"ive
approaches that combine techniques from safe RL and offline RL can only learn
sub-optimal solutions. We thus develop a simple yet effective algorithm,
Constraints Penalized Q-Learning (CPQ), to solve the problem. Our method admits
the use of data generated by mixed behavior policies. We present a theoretical
analysis and demonstrate empirically that our approach can learn robustly
across a variety of benchmark control tasks, outperforming several baselines.

7
The categories of this paper are: hep-th
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
29
The categories of this paper are: funct-an
16
The categories of this paper are: stat
8
The categories of this paper are: hep-ex
20
The categories of this paper are: acc-phys
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
37
The categories of this paper are: supr-con