Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Detecting Dead Weights and Units in Neural Networks   Deep Neural Networks are highly over-parameterized and the size of the neural
networks can be reduced significantly after training without any decrease in
performance. One can clearly see this phenomenon in a wide range of
architectures trained for various problems. Weight/channel pruning,
distillation, quantization, matrix factorization are some of the main methods
one can use to remove the redundancy to come up with smaller and faster models.
  This work starts with a short informative chapter, where we motivate the
pruning idea and provide the necessary notation. In the second chapter, we
compare various saliency scores in the context of parameter pruning. Using the
insights obtained from this comparison and stating the problems it brings we
motivate why pruning units instead of the individual parameters might be a
better idea. We propose some set of definitions to quantify and analyze units
that don't learn and create any useful information. We propose an efficient way
for detecting dead units and use it to select which units to prune. We get 5x
model size reduction through unit-wise pruning on MNIST.

13
The categories of this paper are: quant-ph
33
The categories of this paper are: plasm-ph
18
The categories of this paper are: econ
8
The categories of this paper are: hep-ex
24
The categories of this paper are: dg-ga
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
11
The categories of this paper are: hep-lat
26
The categories of this paper are: bayes-an
34
The categories of this paper are: chem-ph
17
The categories of this paper are: q-fin
10
The categories of this paper are: q-bio
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
4
The categories of this paper are: cond-mat
19
The categories of this paper are: eess