Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Finite-State Controllers for Partially Observable Environments   Reactive (memoryless) policies are sufficient in completely observable Markov
decision processes (MDPs), but some kind of memory is usually necessary for
optimal control of a partially observable MDP. Policies with finite memory can
be represented as finite-state automata. In this paper, we extend Baird and
Moore's VAPS algorithm to the problem of learning general finite-state
automata. Because it performs stochastic gradient descent, this algorithm can
be shown to converge to a locally optimal finite-state controller. We provide
the details of the algorithm and then consider the question of under what
conditions stochastic gradient descent will outperform exact gradient descent.
We conclude with empirical results comparing the performance of stochastic and
exact gradient descent, and showing the ability of our algorithm to extract the
useful information contained in the sequence of past observations to compensate
for the lack of observability at each time-step.

0
The categories of this paper are: hep-ph
5
The categories of this paper are: gr-qc
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
33
The categories of this paper are: plasm-ph
27
The categories of this paper are: comp-gas
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
35
The categories of this paper are: mtrl-th
30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
9
The categories of this paper are: nlin
11
The categories of this paper are: hep-lat