Decision-theoretic agents predict and evaluate the results of their actions using a model, or ontology, of their environment. An agent's goal, or utility function, may also be specified in terms of the states of, or entities within, its ontology. If the agent may upgrade or replace its ontology, it faces a crisis: the agent's original goal may not be well-defined with respect to its new ontology. This crisis must be resolved before the agent can make plans towards achieving its goals.

We discuss in this paper which sorts of agents will undergo ontological crises and why we may want to create such agents. We present some concrete examples, and argue that a well-defined procedure for resolving ontological crises is needed.

We point to some possible approaches to solving this problem, and evaluate these methods on our examples.

## 2. Introduction: Goals And Utility Functions

An agent is any person or thing that performs actions in order to achieve a goal. These goals may involve anything of which the agent is aware, from its own inputs and outputs to distant physical objects. When creating an artificial agent, it is natural to be interested in which goals we choose to give it. When we create something, we usually do so because we expect it to be useful to us. Thus the goals we give to artificial agents should be things that we want to see accomplished.

Programmers of artificial agents are then faced with the task of specifying a goal.

In our discussion we assume that goals take the form of utility functions defined on the set of possible states within the agent's ontology. If a programmer is specifying a utility function "by hand" - that is, by looking at the ontology and directly assigning utilities to different states - then the ontology must be comprehensible to the programmer. This will typically be the case for an ontology that the programmer has designed, but not necessarily so for one that an agent has learned from experience.

An agent with a fixed ontology is not a very powerful agent, so we would like to discuss an agent that begins with an ontology that its programmers understand and have specified a utility function over, and then upgrades or replaces its ontology. If the agent's utility function is defined in terms of states of, or objects within, its initial ontology, then it cannot evaluate utilities within its new ontology unless it translates its utility function somehow.

Consider, for example, an agent schooled in classical physics. Perhaps this agent has a goal that is easy to specify in terms of the movement of atoms, such as to maintain a particular temperature within a given region of space. If we replace our agent's ontology with a quantum one, it is no longer obvious how the agent should evaluate the desirability of a given state. If its utility function is determined by temperature, and temperature is determined by the movement of atoms, then the agent's utility function is determined by the movement of atoms. Yet in a quantum worldview, atoms are not clearly-defined objects. Atoms are not even fundamental to a quantum worldview, so the agent's ontology may contain no reference to atoms whatsoever. How then, can the agent define its utility function?

One way to sidestep the problem of ontological crises is to define the agent's utility function entirely in terms of its percepts, as the set of possible perceptsequences is one aspect of the agent's ontology that does not change. Marcus Hutter's universal agent AIXI [1] uses this approach, and always tries to maximize