Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Unusual Effectiveness of Averaging in GAN Training   We examine two different techniques for parameter averaging in GAN training.
Moving Average (MA) computes the time-average of parameters, whereas
Exponential Moving Average (EMA) computes an exponentially discounted sum.
Whilst MA is known to lead to convergence in bilinear settings, we provide the
-- to our knowledge -- first theoretical arguments in support of EMA. We show
that EMA converges to limit cycles around the equilibrium with vanishing
amplitude as the discount parameter approaches one for simple bilinear games
and also enhances the stability of general GAN training. We establish
experimentally that both techniques are strikingly effective in the
non-convex-concave GAN setting as well. Both improve inception and FID scores
on different architectures and for different GAN objectives. We provide
comprehensive experimental results across a range of datasets -- mixture of
Gaussians, CIFAR-10, STL-10, CelebA and ImageNet -- to demonstrate its
effectiveness. We achieve state-of-the-art results on CIFAR-10 and produce
clean CelebA face images.\footnote{~The code is available at
\url{https://github.com/yasinyazici/EMA_GAN}}

2
The categories of this paper are: math
21
The categories of this paper are: adap-org
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
0
The categories of this paper are: hep-ph
31
The categories of this paper are: ao-sci
23
The categories of this paper are: solv-int
27
The categories of this paper are: comp-gas
30
The categories of this paper are: q-alg
25
The categories of this paper are: patt-sol
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
34
The categories of this paper are: chem-ph
12
The categories of this paper are: nucl-th