Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Modeling Recurrence for Transformer   Recently, the Transformer model that is based solely on attention mechanisms,
has advanced the state-of-the-art on various machine translation tasks.
However, recent studies reveal that the lack of recurrence hinders its further
improvement of translation capacity. In response to this problem, we propose to
directly model recurrence for Transformer with an additional recurrence
encoder. In addition to the standard recurrent neural network, we introduce a
novel attentive recurrent network to leverage the strengths of both attention
and recurrent networks. Experimental results on the widely-used WMT14
English-German and WMT17 Chinese-English translation tasks demonstrate the
effectiveness of the proposed approach. Our studies also reveal that the
proposed model benefits from a short-cut that bridges the source and target
sequences with a single recurrent layer, which outperforms its deep
counterpart.

33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
30
The categories of this paper are: q-alg
32
The categories of this paper are: atom-ph
22
The categories of this paper are: chao-dyn
16
The categories of this paper are: stat
24
The categories of this paper are: dg-ga
12
The categories of this paper are: nucl-th
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
3
The categories of this paper are: physics
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
27
The categories of this paper are: comp-gas
35
The categories of this paper are: mtrl-th