Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Block-wise Dynamic Sparseness   Neural networks have achieved state of the art performance across a wide
variety of machine learning tasks, often with large and computation-heavy
models. Inducing sparseness as a way to reduce the memory and computation
footprint of these models has seen significant research attention in recent
years. In this paper, we present a new method for \emph{dynamic sparseness},
whereby part of the computations are omitted dynamically, based on the input.
For efficiency, we combined the idea of dynamic sparseness with block-wise
matrix-vector multiplications. In contrast to static sparseness, which
permanently zeroes out selected positions in weight matrices, our method
preserves the full network capabilities by potentially accessing any trained
weights. Yet, matrix vector multiplications are accelerated by omitting a
pre-defined fraction of weight blocks from the matrix, based on the input.
Experimental results on the task of language modeling, using recurrent and
quasi-recurrent models, show that the proposed method can outperform a
magnitude-based static sparseness baseline. In addition, our method achieves
similar language modeling perplexities as the dense baseline, at half the
computational cost at inference time.

36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
8
The categories of this paper are: hep-ex
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
30
The categories of this paper are: q-alg
13
The categories of this paper are: quant-ph
29
The categories of this paper are: funct-an
15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
7
The categories of this paper are: hep-th
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
24
The categories of this paper are: dg-ga
0
The categories of this paper are: hep-ph