  There has been substantial progress on finding game-theoretic equilibria.
Most of that work has focused on games with finite, discrete action spaces.
However, many games involving space, time, money, and other fine-grained
quantities have continuous action spaces (or are best modeled as such). We
study the problem of finding an approximate Nash equilibrium of games with
continuous action sets. The standard measure of closeness to Nash equilibrium
is exploitability, which measures how much players can benefit from
unilaterally changing their strategy. We propose two new methods that minimize
an approximation of the exploitability with respect to the strategy profile.
The first method uses a learned best-response function, which takes the current
strategy profile as input and returns candidate best responses for each player.
The strategy profile and best-response functions are trained simultaneously,
with the former trying to minimize exploitability while the latter tries to
maximize it. The second method maintains an ensemble of candidate best
responses for each player. In each iteration, the best-performing elements of
each ensemble are used to update the current strategy profile. The strategy
profile and best-response ensembles are simultaneously trained to minimize and
maximize the approximate exploitability, respectively. We evaluate our methods
on various continuous games, showing that they outperform prior methods.
