  Consider the following two-person mixed strategy game of a probabilist
against Nature with respect to the parameters $(f, \mathcal{B},\pi)$, where $f$
is a convex function satisfying certain regularity conditions, $\mathcal{B}$ is
either the set $\{L_i\}_{i=1}^n$ or its convex hull with each $L_i$ being a
Markov infinitesimal generator on a finite state space $\mathcal{X}$ and $\pi$
is a given positive discrete distribution on $\mathcal{X}$. The probabilist
chooses a prior measure $\mu$ within the set of probability measures on
$\mathcal{B}$ denoted by $\mathcal{P}(\mathcal{B})$ and picks a $L \in
\mathcal{B}$ at random according to $\mu$, whereas Nature follows a pure
strategy to select $M \in \mathcal{L}(\pi)$, the set of $\pi$-reversible Markov
generators on $\mathcal{X}$. Nature pays an amount $D_f(M||L)$, the
$f$-divergence from $L$ to $M$, to the probabilist. We prove that a mixed
strategy Nash equilibrium always exists, and establish a minimax result on the
expected payoff of the game. This also contrasts with the pure strategy version
of the game where we show a Nash equilibrium may not exist. To find
approximately a mixed strategy Nash equilibrium, we propose and develop a
simple projected subgradient algorithm that provably converges with a rate of
$\mathcal{O}(1/\sqrt{t})$, where $t$ is the number of iterations. In addition,
we elucidate the relationships of Nash equilibrium with other seemingly
disparate notions such as weighted information centroid, Chebyshev center and
Bayes risk. This article generalizes the two-person game of a statistician
against Nature developed in the literature, and highlights the powerful
interplay and synergy between modern Markov chains theory and geometry,
information theory, game theory, optimization and mathematical statistics.
