

![0_image_0.png](0_image_0.png)

of ˆrk+2 will potentially depend not only on ak, but also on ak+1, the agent assumes that ak+1 is chosen to simply maximise the estimated reward ˆrk+2.

The π 2back agent can see that by missing out on the resting reward of 2−k−4for one cycle and climbing, a greater reward of 2−k will be had when sliding back down the slide in the following cycle.

By definition π 2forward generalises π 2back in a way that more closely reflects the value function V and thus in general V
π 2forward µ > V π 2back µ. It then follows that Υ(π 2forward) >
Υ(π 2back) as we would intuitively expect for this more powerful agent.

In a similar way agents of increasing complexity and adaptability can be defined which will have still greater intelligence. However with more complex agents it is usually difficult to theoretically establish whether one agent has more or less universal intelligence than another. Nevertheless, in the simple examples above we saw that the more flexible and powerful an agent was, the higher its universal intelligence.

A very intelligent agent. A very smart agent would perform well in simple environments, and reasonably well compared to most other agents in more complex environments.

From the equation for universal intelligence this would clearly produce a very high value for Υ. Conversely, if Υ was very high then the equation for Υ implies that the agent must perform well in most simple environments and reasonably well in many complex ones also.

A super intelligent agent. Consider what would be required to maximise the value of Υ. By definition, a "perfect" agent would always pick the action which had greatest expected future reward. To do this, for every environment µ ∈ E the agent must take into account how likely it is that it is facing µ given the interaction history so far, and the prior probability of µ, that is, 2−K(µ). It would then consider all possible future interactions that might occur, and how likely they are, and from this select the action in the current cycle that maximises the expected future reward.

This perfect theoretical agent is known as AIXI. It has been precisely defined and studied at length in [Hut05] (see [Hut07b] for a shorter exposition). The connection between universal intelligence and AIXI is not coincidental: Υ was originally derived from the so called "intelligence order relation" (see Definition 5.14 in [Hut05]), which in