Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: meProp: Sparsified Back Propagation for Accelerated Deep Learning with
  Reduced Overfitting   We propose a simple yet effective technique for neural network learning. The
forward propagation is computed as usual. In back propagation, only a small
subset of the full gradient is computed to update the model parameters. The
gradient vectors are sparsified in such a way that only the top-$k$ elements
(in terms of magnitude) are kept. As a result, only $k$ rows or columns
(depending on the layout) of the weight matrix are modified, leading to a
linear reduction ($k$ divided by the vector dimension) in the computational
cost. Surprisingly, experimental results demonstrate that we can update only
1-4% of the weights at each back propagation pass. This does not result in a
larger number of training iterations. More interestingly, the accuracy of the
resulting models is actually improved rather than degraded, and a detailed
analysis is given. The code is available at https://github.com/lancopku/meProp

14
The categories of this paper are: math-ph
19
The categories of this paper are: eess
29
The categories of this paper are: funct-an
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
18
The categories of this paper are: econ
20
The categories of this paper are: acc-phys
21
The categories of this paper are: adap-org
2
The categories of this paper are: math
16
The categories of this paper are: stat
36
The categories of this paper are: cmp-lg
13
The categories of this paper are: quant-ph
15
The categories of this paper are: nucl-ex
30
The categories of this paper are: q-alg
12
The categories of this paper are: nucl-th