Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural
  Machine Translation   Tying the weights of the target word embeddings with the target word
classifiers of neural machine translation models leads to faster training and
often to better translation quality. Given the success of this parameter
sharing, we investigate other forms of sharing in between no sharing and hard
equality of parameters. In particular, we propose a structure-aware output
layer which captures the semantic structure of the output space of words within
a joint input-output embedding. The model is a generalized form of weight tying
which shares parameters but allows learning a more flexible relationship with
input word embeddings and allows the effective capacity of the output layer to
be controlled. In addition, the model shares weights across output classifiers
and translation contexts which allows it to better leverage prior knowledge
about them. Our evaluation on English-to-Finnish and English-to-German datasets
shows the effectiveness of the method against strong encoder-decoder baselines
trained with or without weight tying.

0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess
20
The categories of this paper are: acc-phys
2
The categories of this paper are: math
16
The categories of this paper are: stat
12
The categories of this paper are: nucl-th
33
The categories of this paper are: plasm-ph
13
The categories of this paper are: quant-ph
25
The categories of this paper are: patt-sol
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin
23
The categories of this paper are: solv-int
22
The categories of this paper are: chao-dyn