Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Neuron Interaction Based Representation Composition for Neural Machine
  Translation   Recent NLP studies reveal that substantial linguistic information can be
attributed to single neurons, i.e., individual dimensions of the representation
vectors. We hypothesize that modeling strong interactions among neurons helps
to better capture complex information by composing the linguistic properties
embedded in individual neurons. Starting from this intuition, we propose a
novel approach to compose representations learned by different components in
neural machine translation (e.g., multi-layer networks or multi-head
attention), based on modeling strong interactions among neurons in the
representation vectors. Specifically, we leverage bilinear pooling to model
pairwise multiplicative interactions among individual neurons, and a low-rank
approximation to make the model computationally feasible. We further propose
extended bilinear pooling to incorporate first-order representations.
Experiments on WMT14 English-German and English-French translation tasks show
that our model consistently improves performances over the SOTA Transformer
baseline. Further analyses demonstrate that our approach indeed captures more
syntactic and semantic information as expected.

17
The categories of this paper are: q-fin
16
The categories of this paper are: stat
19
The categories of this paper are: eess
30
The categories of this paper are: q-alg
0
The categories of this paper are: hep-ph
29
The categories of this paper are: funct-an
2
The categories of this paper are: math
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
20
The categories of this paper are: acc-phys
3
The categories of this paper are: physics
31
The categories of this paper are: ao-sci
23
The categories of this paper are: solv-int