Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reducing the Model Order of Deep Neural Networks Using Information
  Theory   Deep neural networks are typically represented by a much larger number of
parameters than shallow models, making them prohibitive for small footprint
devices. Recent research shows that there is considerable redundancy in the
parameter space of deep neural networks. In this paper, we propose a method to
compress deep neural networks by using the Fisher Information metric, which we
estimate through a stochastic optimization method that keeps track of
second-order information in the network. We first remove unimportant parameters
and then use non-uniform fixed point quantization to assign more bits to
parameters with higher Fisher Information estimates. We evaluate our method on
a classification task with a convolutional neural network trained on the MNIST
data set. Experimental results show that our method outperforms existing
methods for both network pruning and quantization.

0
The categories of this paper are: hep-ph
22
The categories of this paper are: chao-dyn
20
The categories of this paper are: acc-phys
34
The categories of this paper are: chem-ph
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
4
The categories of this paper are: cond-mat
25
The categories of this paper are: patt-sol
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom
36
The categories of this paper are: cmp-lg
19
The categories of this paper are: eess