  In this paper we study a mean field model for discrete time, finite number of
states, dynamic games. These models arise in situations that involve a very
large number of agents moving from state to state according to certain
optimality criteria. The mean field approach for optimal control and
differential games (continuous state and time) was introduced by Lasry and
Lions. Here we consider a discrete version of the problem. Our setting is the
following: we assume that there is a very large number of identical agents
which can be in a finite number of states. Because the number of agents is very
large, we assume the mean field hypothesis, that is, that the only relevant
information for the global evolution is the fraction $\pi^n_i$ of players in
each state $i$ at time $n$. The agents look for minimizing a running cost,
which depends on $\pi$, plus a terminal cost $V^N$. In contrast with optimal
control, where usually only the terminal cost $V^N$ is necessary to solve the
problem, in mean-field games both the initial distribution of agents $\pi^0$
and the terminal cost $V^N$ are necessary to determine the solutions, that is,
the distribution of players $\pi^n$ and value function $V^n$, for $0\leq n\leq
N$. Because both initial and terminal data needs to be specified, we call this
problem the initial-terminal value problem. Existence of solutions is
non-trivial. Nevertheless, following the ideas of Lasry and Lions, we were able
to establish existence and uniqueness, both for the stationary and for the
initial-terminal value problems. In the last part of the paper we prove the
main result of the paper, namely the exponential convergence to a stationary
solution of $(\pi^0, V^0)$, as $N\to \infty$, for the initial-terminal value
problem with (fixed) data $\pi^{-N}$ and $V^N$.
