Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Structured Ensembles: an Approach to Reduce the Memory Footprint of
  Ensemble Methods   In this paper, we propose a novel ensembling technique for deep neural
networks, which is able to drastically reduce the required memory compared to
alternative approaches. In particular, we propose to extract multiple
sub-networks from a single, untrained neural network by solving an end-to-end
optimization task combining differentiable scaling over the original
architecture, with multiple regularization terms favouring the diversity of the
ensemble. Since our proposal aims to detect and extract sub-structures, we call
it Structured Ensemble. On a large experimental evaluation, we show that our
method can achieve higher or comparable accuracy to competing methods while
requiring significantly less storage. In addition, we evaluate our ensembles in
terms of predictive calibration and uncertainty, showing they compare
favourably with the state-of-the-art. Finally, we draw a link with the
continual learning literature, and we propose a modification of our framework
to handle continuous streams of tasks with a sub-linear memory cost. We compare
with a number of alternative strategies to mitigate catastrophic forgetting,
highlighting advantages in terms of average accuracy and memory.

32
The categories of this paper are: atom-ph
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
29
The categories of this paper are: funct-an
11
The categories of this paper are: hep-lat
30
The categories of this paper are: q-alg
19
The categories of this paper are: eess
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
8
The categories of this paper are: hep-ex
22
The categories of this paper are: chao-dyn