Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Successfully Applying the Stabilized Lottery Ticket Hypothesis to the
  Transformer Architecture   Sparse models require less memory for storage and enable a faster inference
by reducing the necessary number of FLOPs. This is relevant both for
time-critical and on-device computations using neural networks. The stabilized
lottery ticket hypothesis states that networks can be pruned after none or few
training iterations, using a mask computed based on the unpruned converged
model. On the transformer architecture and the WMT 2014 English-to-German and
English-to-French tasks, we show that stabilized lottery ticket pruning
performs similar to magnitude pruning for sparsity levels of up to 85%, and
propose a new combination of pruning techniques that outperforms all other
techniques for even higher levels of sparsity. Furthermore, we confirm that the
parameter's initial sign and not its specific value is the primary factor for
successful training, and show that magnitude pruning could be used to find
winning lottery tickets.

10
The categories of this paper are: q-bio
19
The categories of this paper are: eess
11
The categories of this paper are: hep-lat
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
18
The categories of this paper are: econ
4
The categories of this paper are: cond-mat
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
14
The categories of this paper are: math-ph
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
13
The categories of this paper are: quant-ph
7
The categories of this paper are: hep-th
8
The categories of this paper are: hep-ex
21
The categories of this paper are: adap-org