Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Frequency-based Search-control in Dyna   Model-based reinforcement learning has been empirically demonstrated as a
successful strategy to improve sample efficiency. In particular, Dyna is an
elegant model-based architecture integrating learning and planning that
provides huge flexibility of using a model. One of the most important
components in Dyna is called search-control, which refers to the process of
generating state or state-action pairs from which we query the model to acquire
simulated experiences. Search-control is critical in improving learning
efficiency. In this work, we propose a simple and novel search-control strategy
by searching high frequency regions of the value function. Our main intuition
is built on Shannon sampling theorem from signal processing, which indicates
that a high frequency signal requires more samples to reconstruct. We
empirically show that a high frequency function is more difficult to
approximate. This suggests a search-control strategy: we should use states from
high frequency regions of the value function to query the model to acquire more
samples. We develop a simple strategy to locally measure the frequency of a
function by gradient and hessian norms, and provide theoretical justification
for this approach. We then apply our strategy to search-control in Dyna, and
conduct experiments to show its property and effectiveness on benchmark
domains.

15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
33
The categories of this paper are: plasm-ph
21
The categories of this paper are: adap-org
19
The categories of this paper are: eess
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
10
The categories of this paper are: q-bio
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
7
The categories of this paper are: hep-th
25
The categories of this paper are: patt-sol
6
The categories of this paper are: astro-ph
18
The categories of this paper are: econ