it receives by learning about the structure of the environment and **the goals it needs** to accomplish in order to receive rewards.

Formally, the process of interaction produces an increasing history o1r1a1o2r2a2o3r3a3o4 . . . of observations o, rewards r ≥ 0, and actions a**. The agent**
is simply a function, denoted by π, which is a probability measure over actions conditioned on the current history, for example, π(a3|o1r1a1o2r2**). The environment,**
denoted µ, is similarly defined: µ(okrk|o1r1a1o2r2a2 . . . ok−1rk−1ak−1). The performance of agent π in environment µ **can be measured by its total expected reward**
V
π µ
:= E[P∞
i=1 ri|µ, π**], called value. The largest interesting class of environments is**
the class E of all computable probability distributions µ**. For technical reasons, the** values are assumed to be bounded by some constant c.

To get a single performance measure V
π µis averaged over all µ ∈ E**. As there**
are an infinite number of environments, with no bound on their complexity, it is impossible to take the expected value with respect to a uniform distribution - some environments must be weighted more heavily than others. Considering the agent's perspective on the problem, it is the same as asking: Given several different hypotheses which are consistent with the observations, which hypothesis should be considered the most likely? This is a fundamental problem in inductive inference for which the standard solution is to invoke Occam's razor: **Given multiple hypotheses**
which are consistent with the data, simpler ones should be preferred. **As this is generally considered the most intelligent thing to do, one should test agents in such a**
way that they are, at least on average, rewarded for correctly applying Occam's razor. This means that the a priori distribution over environments should be weighted towards simpler environments.

As each environment µ **is described by a computable measure, their complexity**
can be measured with Kolmogorov complexity K(µ**), which is simply the length of**
the shortest program that computes µ [21]. The right a priori weight for µ **is 2**−K(µ).

We can now define the universal intelligence of an agent π **to simply be its expected**
performance,

$$\Upsilon(\pi):=\sum_{\mu\in E}2^{-K(\mu)}V_{\mu}^{\pi}.$$

By construction, universal intelligence measures the general ability of an agent to perform well in a very wide range of environments, similar to the essence of many informal definitions of intelligence [18]. The definition places no restrictions on the internal workings of the agent; it only requires that the agent **is capable of** generating output and receiving input which includes a reward signal. **If we wish** to bias the test to reflect world knowledge then we can condition the **complexity**
measure. For example, use K(µ|D) where D **is some set of background knowledge**
such as Wikipedia.

By considering V
π µ**for a number of basic environments, such as small MDPs,**
and agents with simple but very general optimisation strategies, it is **clear that**
Υ correctly orders the relative intelligence of these agents in a natural way. A
very high value of Υ would imply that an agent is able to perform well in many