  We introduce $\alpha$-Rank, a principled evolutionary dynamics methodology
for the evaluation and ranking of agents in large-scale multi-agent
interactions, grounded in a novel dynamical game-theoretic solution concept
called Markov-Conley chains (MCCs). The approach leverages continuous- and
discrete-time evolutionary dynamical systems applied to empirical games, and
scales tractably in the number of agents, the type of interactions, and the
type of empirical games (symmetric and asymmetric). Current models are
fundamentally limited in one or more of these dimensions and are not guaranteed
to converge to the desired game-theoretic solution concept (typically the Nash
equilibrium). $\alpha$-Rank provides a ranking over the set of agents under
evaluation and provides insights into their strengths, weaknesses, and
long-term dynamics. This is a consequence of the links we establish to the MCC
solution concept when the underlying evolutionary model's ranking-intensity
parameter, $\alpha$, is chosen to be large, which exactly forms the basis of
$\alpha$-Rank. In contrast to the Nash equilibrium, which is a static concept
based on fixed points, MCCs are a dynamical solution concept based on the
Markov chain formalism, Conley's Fundamental Theorem of Dynamical Systems, and
the core ingredients of dynamical systems: fixed points, recurrent sets,
periodic orbits, and limit cycles. $\alpha$-Rank runs in polynomial time with
respect to the total number of pure strategy profiles, whereas computing a Nash
equilibrium for a general-sum game is known to be intractable. We introduce
proofs that not only provide a unifying perspective of existing continuous- and
discrete-time evolutionary evaluation models, but also reveal the formal
underpinnings of the $\alpha$-Rank methodology. We empirically validate the
method in several domains including AlphaGo, AlphaZero, MuJoCo Soccer, and
Poker.
