Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Filter Pre-Pruning for Improved Fine-tuning of Quantized Deep Neural
  Networks   Deep Neural Networks(DNNs) have many parameters and activation data, and
these both are expensive to implement. One method to reduce the size of the DNN
is to quantize the pre-trained model by using a low-bit expression for weights
and activations, using fine-tuning to recover the drop in accuracy. However, it
is generally difficult to train neural networks which use low-bit expressions.
One reason is that the weights in the middle layer of the DNN have a wide
dynamic range and so when quantizing the wide dynamic range into a few bits,
the step size becomes large, which leads to a large quantization error and
finally a large degradation in accuracy. To solve this problem, this paper
makes the following three contributions without using any additional learning
parameters and hyper-parameters. First, we analyze how batch normalization,
which causes the aforementioned problem, disturbs the fine-tuning of the
quantized DNN. Second, based on these results, we propose a new pruning method
called Pruning for Quantization (PfQ) which removes the filters that disturb
the fine-tuning of the DNN while not affecting the inferred result as far as
possible. Third, we propose a workflow of fine-tuning for quantized DNNs using
the proposed pruning method(PfQ). Experiments using well-known models and
datasets confirmed that the proposed method achieves higher performance with a
similar model size than conventional quantization methods including
fine-tuning.

8
The categories of this paper are: hep-ex
20
The categories of this paper are: acc-phys
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
24
The categories of this paper are: dg-ga
25
The categories of this paper are: patt-sol
10
The categories of this paper are: q-bio
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
30
The categories of this paper are: q-alg
17
The categories of this paper are: q-fin
2
The categories of this paper are: math
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom