Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several
  Thinner Ones?   One of the generally accepted views of modern deep learning is that
increasing the number of parameters usually leads to better quality. The two
easiest ways to increase the number of parameters is to increase the size of
the network, e.g. width, or to train a deep ensemble; both approaches improve
the performance in practice. In this work, we consider a fixed memory budget
setting, and investigate, what is more effective: to train a single wide
network, or to perform a memory split -- to train an ensemble of several
thinner networks, with the same total number of parameters? We find that, for
large enough budgets, the number of networks in the ensemble, corresponding to
the optimal memory split, is usually larger than one. Interestingly, this
effect holds for the commonly used sizes of the standard architectures. For
example, one WideResNet-28-10 achieves significantly worse test accuracy on
CIFAR-100 than an ensemble of sixteen thinner WideResNets: 80.6% and 82.52%
correspondingly. We call the described effect the Memory Split Advantage and
show that it holds for a variety of datasets and model architectures.

4
The categories of this paper are: cond-mat
5
The categories of this paper are: gr-qc
32
The categories of this paper are: atom-ph
3
The categories of this paper are: physics
2
The categories of this paper are: math
26
The categories of this paper are: bayes-an
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
33
The categories of this paper are: plasm-ph
0
The categories of this paper are: hep-ph
7
The categories of this paper are: hep-th
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
24
The categories of this paper are: dg-ga
22
The categories of this paper are: chao-dyn