  We propose Turing Learning, a novel system identification method for
inferring the behavior of natural or artificial systems. Turing Learning
simultaneously optimizes two populations of computer programs, one representing
models of the behavior of the system under investigation, and the other
representing classifiers. By observing the behavior of the system as well as
the behaviors produced by the models, two sets of data samples are obtained.
The classifiers are rewarded for discriminating between these two sets, that
is, for correctly categorizing data samples as either genuine or counterfeit.
Conversely, the models are rewarded for 'tricking' the classifiers into
categorizing their data samples as genuine. Unlike other methods for system
identification, Turing Learning does not require predefined metrics to quantify
the difference between the system and its models. We present two case studies
with swarms of simulated robots and prove that the underlying behaviors cannot
be inferred by a metric-based system identification method. By contrast, Turing
Learning infers the behaviors with high accuracy. It also produces a useful
by-product - the classifiers - that can be used to detect abnormal behavior in
the swarm. Moreover, we show that Turing Learning also successfully infers the
behavior of physical robot swarms. The results show that collective behaviors
can be directly inferred from motion trajectories of individuals in the swarm,
which may have significant implications for the study of animal collectives.
Furthermore, Turing Learning could prove useful whenever a behavior is not
easily characterizable using metrics, making it suitable for a wide range of
applications.
