Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Temporal Concatenation for Markov Decision Processes   We propose and analyze a Temporal Concatenation (TC) heuristic for solving
large-scale finite-horizon Markov decision processes (MDP). The Temporal
Concatenation divides a finite-horizon MDP into smaller sub-problems along the
time horizon, and generates an overall solution by simply concatenating the
optimal solutions from these sub-problems. As a "black box" architecture,
Temporal Concatenation works with a wide range of existing MDP algorithms with
the potential of substantial speed-up at the expense of minor performance
degradation. Our main results characterize the regret of Temporal
Concatenation, defined as the gap between the expected rewards from Temporal
Concatenation's solution and that from the optimal solution. We provide upper
bounds that show, when the underlying MDP satisfies a bounded-diameter
criterion, the regret of Temporal Concatenation is bounded by a constant
independent of the length of the horizon. Conversely, we provide matching lower
bounds that demonstrate that, for any finite diameter, there exist MDP
instances for which the regret upper bound is tight. We further contextualize
the theoretical results in an illustrative example of dynamic energy management
with storage, and provide simulation results to assess Temporal Concatenation's
average-case regret within a family of MDPs related to graph traversal.

8
The categories of this paper are: hep-ex
34
The categories of this paper are: chem-ph
36
The categories of this paper are: cmp-lg
23
The categories of this paper are: solv-int
16
The categories of this paper are: stat
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
35
The categories of this paper are: mtrl-th
21
The categories of this paper are: adap-org
7
The categories of this paper are: hep-th
19
The categories of this paper are: eess
28
The categories of this paper are: alg-geom
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
4
The categories of this paper are: cond-mat
25
The categories of this paper are: patt-sol