Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Safe Exploration for Constrained Reinforcement Learning with Provable
  Guarantees   We consider the problem of learning an episodic safe control policy that
minimizes an objective function, while satisfying necessary safety constraints
-- both during learning and deployment. We formulate this safety constrained
reinforcement learning (RL) problem using the framework of a finite-horizon
Constrained Markov Decision Process (CMDP) with an unknown transition
probability function. Here, we model the safety requirements as constraints on
the expected cumulative costs that must be satisfied during all episodes of
learning. We propose a model-based safe RL algorithm that we call the
Optimistic-Pessimistic Safe Reinforcement Learning (OPSRL) algorithm, and show
that it achieves an $\tilde{\mathcal{O}}(S^{2}\sqrt{A H^{7}K}/ (\bar{C} -
\bar{C}_{b}))$ cumulative regret without violating the safety constraints
during learning, where $S$ is the number of states, $A$ is the number of
actions, $H$ is the horizon length, $K$ is the number of learning episodes, and
$(\bar{C} - \bar{C}_{b})$ is the safety gap, i.e., the difference between the
constraint value and the cost of a known safe baseline policy. The scaling as
$\tilde{\mathcal{O}}(\sqrt{K})$ is the same as the traditional approach where
constraints may be violated during learning, which means that our algorithm
suffers no additional regret in spite of providing a safety guarantee. Our key
idea is to use an optimistic exploration approach with pessimistic constraint
enforcement for learning the policy. This approach simultaneously incentivizes
the exploration of unknown states while imposing a penalty for visiting states
that are likely to cause violation of safety constraints. We validate our
algorithm by evaluating its performance on benchmark problems against
conventional approaches.

18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
35
The categories of this paper are: mtrl-th
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
9
The categories of this paper are: nlin
25
The categories of this paper are: patt-sol
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
4
The categories of this paper are: cond-mat
14
The categories of this paper are: math-ph
16
The categories of this paper are: stat
13
The categories of this paper are: quant-ph