Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Self-Modification of Policy and Utility Function in Rational Agents   Any agent that is part of the environment it interacts with and has versatile
actuators (such as arms and fingers), will in principle have the ability to
self-modify -- for example by changing its own source code. As we continue to
create more and more intelligent agents, chances increase that they will learn
about this ability. The question is: will they want to use it? For example,
highly intelligent systems may find ways to change their goals to something
more easily achievable, thereby `escaping' the control of their designers. In
an important paper, Omohundro (2008) argued that goal preservation is a
fundamental drive of any intelligent system, since a goal is more likely to be
achieved if future versions of the agent strive towards the same goal. In this
paper, we formalise this argument in general reinforcement learning, and
explore situations where it fails. Our conclusion is that the self-modification
possibility is harmless if and only if the value function of the agent
anticipates the consequences of self-modifications and use the current utility
function when evaluating the future.

2
The categories of this paper are: math
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
4
The categories of this paper are: cond-mat
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
20
The categories of this paper are: acc-phys
15
The categories of this paper are: nucl-ex
13
The categories of this paper are: quant-ph
7
The categories of this paper are: hep-th
37
The categories of this paper are: supr-con
24
The categories of this paper are: dg-ga
10
The categories of this paper are: q-bio
8
The categories of this paper are: hep-ex
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas