Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Lottery Jackpots Exist in Pre-trained Models   Network pruning is an effective approach to reduce network complexity with
acceptable performance compromise. Existing studies achieve the sparsity of
neural networks via time-consuming weight tuning or complex search on networks
with expanded width, which greatly limits the applications of network pruning.
In this paper, we show that high-performing and sparse sub-networks without the
involvement of weight tuning, termed "lottery jackpots", exist in pre-trained
models with unexpanded width. For example, we obtain a lottery jackpot that has
only 10% parameters and still reaches the performance of the original dense
VGGNet-19 without any modifications on the pre-trained weights on CIFAR-10.
Furthermore, we observe that the sparse masks derived from many existing
pruning criteria have a high overlap with the searched mask of our lottery
jackpot, among which, the magnitude-based pruning results in the most similar
mask with ours. Based on this insight, we initialize our sparse mask using the
magnitude-based pruning, resulting in at least 3x cost reduction on the lottery
jackpot search while achieving comparable or even better performance.
Specifically, our magnitude-based lottery jackpot removes 90% weights in
ResNet-50, while it easily obtains more than 70% top-1 accuracy using only 10
searching epochs on ImageNet. Our project is available at
https://github.com/lottery-jackpot/lottery-jackpot.

2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
7
The categories of this paper are: hep-th
10
The categories of this paper are: q-bio
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
4
The categories of this paper are: cond-mat
24
The categories of this paper are: dg-ga
29
The categories of this paper are: funct-an
28
The categories of this paper are: alg-geom
22
The categories of this paper are: chao-dyn
0
The categories of this paper are: hep-ph