Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reward Tampering Problems and Solutions in Reinforcement Learning: A
  Causal Influence Diagram Perspective   Can humans get arbitrarily capable reinforcement learning (RL) agents to do
their bidding? Or will sufficiently capable RL agents always find ways to
bypass their intended objectives by shortcutting their reward signal? This
question impacts how far RL can be scaled, and whether alternative paradigms
must be developed in order to build safe artificial general intelligence. In
this paper, we study when an RL agent has an instrumental goal to tamper with
its reward process, and describe design principles that prevent instrumental
goals for two different types of reward tampering (reward function tampering
and RF-input tampering). Combined, the design principles can prevent both types
of reward tampering from being instrumental goals. The analysis benefits from
causal influence diagrams to provide intuitive yet precise formalizations.

28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
4
The categories of this paper are: cond-mat
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
11
The categories of this paper are: hep-lat
15
The categories of this paper are: nucl-ex
14
The categories of this paper are: math-ph
29
The categories of this paper are: funct-an
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
36
The categories of this paper are: cmp-lg
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
17
The categories of this paper are: q-fin