Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Near-Optimal Reward-Free Exploration for Linear Mixture MDPs with
  Plug-in Solver   Although model-based reinforcement learning (RL) approaches are considered
more sample efficient, existing algorithms are usually relying on sophisticated
planning algorithm to couple tightly with the model-learning procedure. Hence
the learned models may lack the ability of being re-used with more specialized
planners. In this paper we address this issue and provide approaches to learn
an RL model efficiently without the guidance of a reward signal. In particular,
we take a plug-in solver approach, where we focus on learning a model in the
exploration phase and demand that \emph{any planning algorithm} on the learned
model can give a near-optimal policy. Specicially, we focus on the linear
mixture MDP setting, where the probability transition matrix is a (unknown)
convex combination of a set of existing models. We show that, by establishing a
novel exploration algorithm, the plug-in approach learns a model by taking
$\tilde{O}(d^2H^3/\epsilon^2)$ interactions with the environment and \emph{any}
$\epsilon$-optimal planner on the model gives an $O(\epsilon)$-optimal policy
on the original model. This sample complexity matches lower bounds for
non-plug-in approaches and is \emph{statistically optimal}. We achieve this
result by leveraging a careful maximum total-variance bound using Bernstein
inequality and properties specified to linear mixture MDP.

37
The categories of this paper are: supr-con
20
The categories of this paper are: acc-phys
30
The categories of this paper are: q-alg
35
The categories of this paper are: mtrl-th
10
The categories of this paper are: q-bio
4
The categories of this paper are: cond-mat
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
36
The categories of this paper are: cmp-lg
29
The categories of this paper are: funct-an
15
The categories of this paper are: nucl-ex
7
The categories of this paper are: hep-th
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex
21
The categories of this paper are: adap-org