Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Pay Less Attention with Lightweight and Dynamic Convolutions   Self-attention is a useful mechanism to build generative models for language
and images. It determines the importance of context elements by comparing each
element to the current time step. In this paper, we show that a very
lightweight convolution can perform competitively to the best reported
self-attention results. Next, we introduce dynamic convolutions which are
simpler and more efficient than self-attention. We predict separate convolution
kernels based solely on the current time-step in order to determine the
importance of context elements. The number of operations required by this
approach scales linearly in the input length, whereas self-attention is
quadratic. Experiments on large-scale machine translation, language modeling
and abstractive summarization show that dynamic convolutions improve over
strong self-attention models. On the WMT'14 English-German test set dynamic
convolutions achieve a new state of the art of 29.7 BLEU.

2
The categories of this paper are: math
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
21
The categories of this paper are: adap-org
32
The categories of this paper are: atom-ph
13
The categories of this paper are: quant-ph
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
18
The categories of this paper are: econ
19
The categories of this paper are: eess