designed for humans, such as the Wechsler Adult Intelligent Scale and Raven Progressive Matrices.

As effective as these tests are for humans, they seem inadequate **for measuring**
machine intelligence as they are highly anthropocentric and embody basic assumptions about the test subject that are likely to be violated by computers. For example, consider the fundamental assumption that the test subject is not simply a collection of specialised algorithms designed only for answering common IQ test **questions.** While this is obviously true of a human, or even an ape, it may not be true of a computer. The computer could be nothing more than a collection of specific algorithms designed to identify patterns in shapes, predict number sequences, write poems on a given subject or solve verbal analogy problems - all things that AI researchers have worked on. Such a machine might be able to obtain **a respectable** IQ score [25], even though outside of these specific test problems it **would be next** to useless. If we try to correct for these limitations by expanding beyond standard tests, as Bringsjord and Schimanski seem to suggest, this once again opens up the difficulty of exactly what, and what not, to test for. Psychometric **AI, at least as it**
is currently formulated, only partially addresses this central question.

## 8 Smith'S Test

The basic structure of Smith's test is that an agent faces a series of problems that are generated by an algorithm [30]. In each iteration the agent must **try to produce** the correct response to the problem that it has been given. The problem generator then responds with a score of how good the agent's answer was. If **the agent so** desires it can submit another answer to the same problem. At some point the agent requests to the problem generator to move onto the next problem **and the score that** the agent received for its last answer to the current problem is then added to its cumulative score. Each interaction cycle counts as one time step and the agent's intelligence is then its total cumulative score considered as a function of time. In order to keep things feasible, the problems must all be in P, i.e. the solution must be verifiable in polynomial time.

We have two main criticisms of Smith's definition. Firstly, while for practical reasons it might make sense to restrict problems to be in P, we do not **see why this**
practical restriction should be a part of the very definition of intelligence as Smith suggests. If some breakthrough meant that agents could solve difficult problems in not just P but sometimes in NP as well, then surely these new agents would be more intelligent?

Secondly, while the definition is somewhat formally defined, it still leaves open the important question of what exactly the tests should be. Smith suggests that researchers should dream up tests and then contribute them to some common pool of tests. As such, this is not a fully specified test.