Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Pruning from Scratch   Network pruning is an important research field aiming at reducing
computational costs of neural networks. Conventional approaches follow a fixed
paradigm which first trains a large and redundant network, and then determines
which units (e.g., channels) are less important and thus can be removed. In
this work, we find that pre-training an over-parameterized model is not
necessary for obtaining the target pruned structure. In fact, a fully-trained
over-parameterized model will reduce the search space for the pruned structure.
We empirically show that more diverse pruned structures can be directly pruned
from randomly initialized weights, including potential models with better
performance. Therefore, we propose a novel network pruning pipeline which
allows pruning from scratch. In the experiments for compressing classification
models on CIFAR10 and ImageNet datasets, our approach not only greatly reduces
the pre-training burden of traditional pruning methods, but also achieves
similar or even higher accuracy under the same computation budgets. Our results
facilitate the community to rethink the effectiveness of existing techniques
used for network pruning.

20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat
32
The categories of this paper are: atom-ph
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
25
The categories of this paper are: patt-sol
9
The categories of this paper are: nlin
31
The categories of this paper are: ao-sci
29
The categories of this paper are: funct-an
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
35
The categories of this paper are: mtrl-th