Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Avoiding Wireheading with Value Reinforcement Learning   How can we design good goals for arbitrarily intelligent agents?
Reinforcement learning (RL) is a natural approach. Unfortunately, RL does not
work well for generally intelligent agents, as RL agents are incentivised to
shortcut the reward sensor for maximum reward -- the so-called wireheading
problem. In this paper we suggest an alternative to RL called value
reinforcement learning (VRL). In VRL, agents use the reward signal to learn a
utility function. The VRL setup allows us to remove the incentive to wirehead
by placing a constraint on the agent's actions. The constraint is defined in
terms of the agent's belief distributions, and does not require an explicit
specification of which actions constitute wireheading.

36
The categories of this paper are: cmp-lg
7
The categories of this paper are: hep-th
24
The categories of this paper are: dg-ga
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
34
The categories of this paper are: chem-ph
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
17
The categories of this paper are: q-fin
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph
3
The categories of this paper are: physics
12
The categories of this paper are: nucl-th