Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Constrained Randomized Shortest-Paths Framework for Optimal
  Exploration   The present work extends the randomized shortest-paths framework (RSP),
interpolating between shortest-path and random-walk routing in a network, in
three directions. First, it shows how to deal with equality constraints on a
subset of transition probabilities and develops a generic algorithm for solving
this constrained RSP problem using Lagrangian duality. Second, it derives a
surprisingly simple iterative procedure to compute the optimal, randomized,
routing policy generalizing the previously developed "soft" Bellman-Ford
algorithm. The resulting algorithm allows balancing exploitation and
exploration in an optimal way by interpolating between a pure random behavior
and the deterministic, optimal, policy (least-cost paths) while satisfying the
constraints. Finally, the two algorithms are applied to Markov decision
problems by considering the process as a constrained RSP on a bipartite
state-action graph. In this context, the derived "soft" value iteration
algorithm appears to be closely related to dynamic policy programming as well
as Kullback-Leibler and path integral control, and similar to a recently
introduced reinforcement learning exploration strategy. This shows that this
strategy is optimal in the RSP sense - it minimizes expected path cost subject
to relative entropy constraint. Simulation results on illustrative examples
show that the model behaves as expected.

32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
28
The categories of this paper are: alg-geom
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
6
The categories of this paper are: astro-ph
19
The categories of this paper are: eess
22
The categories of this paper are: chao-dyn
2
The categories of this paper are: math
35
The categories of this paper are: mtrl-th
13
The categories of this paper are: quant-ph
5
The categories of this paper are: gr-qc