Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Sparse Training via Boosting Pruning Plasticity with Neuroregeneration   Works on lottery ticket hypothesis (LTH) and single-shot network pruning
(SNIP) have raised a lot of attention currently on post-training pruning
(iterative magnitude pruning), and before-training pruning (pruning at
initialization). The former method suffers from an extremely large computation
cost and the latter usually struggles with insufficient performance. In
comparison, during-training pruning, a class of pruning methods that
simultaneously enjoys the training/inference efficiency and the comparable
performance, temporarily, has been less explored. To better understand
during-training pruning, we quantitatively study the effect of pruning
throughout training from the perspective of pruning plasticity (the ability of
the pruned networks to recover the original performance). Pruning plasticity
can help explain several other empirical observations about neural network
pruning in literature. We further find that pruning plasticity can be
substantially improved by injecting a brain-inspired mechanism called
neuroregeneration, i.e., to regenerate the same number of connections as
pruned. We design a novel gradual magnitude pruning (GMP) method, named gradual
pruning with zero-cost neuroregeneration (\textbf{GraNet}), that advances state
of the art. Perhaps most impressively, its sparse-to-sparse version for the
first time boosts the sparse-to-sparse training performance over various
dense-to-sparse methods with ResNet-50 on ImageNet without extending the
training time. We release all codes in
https://github.com/Shiweiliuiiiiiii/GraNet.

15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin
34
The categories of this paper are: chem-ph
21
The categories of this paper are: adap-org
36
The categories of this paper are: cmp-lg
35
The categories of this paper are: mtrl-th
6
The categories of this paper are: astro-ph