Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Sample Complexity of Policy Gradient Finding Second-Order Stationary
  Points   The goal of policy-based reinforcement learning (RL) is to search the maximal
point of its objective. However, due to the inherent non-concavity of its
objective, convergence to a first-order stationary point (FOSP) can not
guarantee the policy gradient methods finding a maximal point. A FOSP can be a
minimal or even a saddle point, which is undesirable for RL. Fortunately, if
all the saddle points are \emph{strict}, all the second-order stationary points
(SOSP) are exactly equivalent to local maxima. Instead of FOSP, we consider
SOSP as the convergence criteria to character the sample complexity of policy
gradient. Our result shows that policy gradient converges to an
$(\epsilon,\sqrt{\epsilon\chi})$-SOSP with probability at least
$1-\widetilde{\mathcal{O}}(\delta)$ after the total cost of
$\mathcal{O}\left(\dfrac{\epsilon^{-\frac{9}{2}}}{(1-\gamma)\sqrt\chi}\log\dfrac{1}{\delta}\right)$,
where $\gamma\in(0,1)$. Our result improves the state-of-the-art result
significantly where it requires
$\mathcal{O}\left(\dfrac{\epsilon^{-9}\chi^{\frac{3}{2}}}{\delta}\log\dfrac{1}{\epsilon\chi}\right)$.
Our analysis is based on the key idea that decomposes the parameter space
$\mathbb{R}^p$ into three non-intersected regions: non-stationary point, saddle
point, and local optimal region, then making a local improvement of the
objective of RL in each region. This technique can be potentially generalized
to extensive policy gradient methods.

6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
5
The categories of this paper are: gr-qc
18
The categories of this paper are: econ
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
21
The categories of this paper are: adap-org
30
The categories of this paper are: q-alg
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph