4 PETER DE BLANC THE SINGULARITY INSTITUTE FOR ARTIFICIAL INTELLIGENCE
like φ
−1 ◦ T
x 0 ◦ φ to approximate T
x 1
. By this we mean that we should be able to predict as accurately as possible the result of some action x in O1 by translating our distribution for the initial state in O1 to a distribution over O0 (using the function φ), predicting the result of action x within O0, and translating this result back to O1 using φ
−1. Similarly, we would like to use O1 to predict the behavior of O0.

Furthermore, we want to to optimize φ and φ
−1so that both models will make similar predictions about sensory data. Thus A0 ◦ φ should be close to A1 and A1 ◦ φ
−1should be close to A0.

To measure distance between two matrices, we treat the columns vectors as probability distributions and sum the Kullback-Leibler divergences of the columns. For two matrices P and Q, let DKL(P||Q) be the sum of the Kullback-Leibler divergences of the columns. When calculating Kullback-Leibler divergence, we consider the columns of the A and T matrices to be the "true" distributions, while those depending on φ or φ
−1 are regarded as the approximations.

So we choose φ and φ
−1to minimize the quantity

$$\left(\sum_{x\in M}D_{K L}(T_{1}^{x}||\phi^{-1}T_{0}^{x}\phi)\right)+D_{K L}(A_{1}||\phi^{-1}A_{0}\phi)$$ $$+\left(\sum_{x\in M}D_{K L}(T_{0}^{x}||\phi T_{1}^{x}\phi^{-1})\right)+D_{K L}(A_{0}||\phi A_{1}\phi^{-1})$$  I shall claim to show that $\phi$ is a $\phi$-function.  
Using a simple hill-climbing algorithm, we have tested our criterion on a simple example.

## 6. Example: The Long Corridor

The agent initially believes that it is standing in a corridor consisting of four discrete locations. The agent's actions are to move left or right. If the agent is already at the end of the corridor and attempts to move further in that direction, it will remain where it is. The agent can see whether it standing at the left end, the right end, or at neither end of the corridor. The agent's goal is to stand at the right end of the corridor.

![0_image_0.png](0_image_0.png)

