Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Reverse Transfer Learning: Can Word Embeddings Trained for Different NLP
  Tasks Improve Neural Language Models?   Natural language processing (NLP) tasks tend to suffer from a paucity of
suitably annotated training data, hence the recent success of transfer learning
across a wide variety of them. The typical recipe involves: (i) training a
deep, possibly bidirectional, neural network with an objective related to
language modeling, for which training data is plentiful; and (ii) using the
trained network to derive contextual representations that are far richer than
standard linear word embeddings such as word2vec, and thus result in important
gains. In this work, we wonder whether the opposite perspective is also true:
can contextual representations trained for different NLP tasks improve language
modeling itself? Since language models (LMs) are predominantly locally
optimized, other NLP tasks may help them make better predictions based on the
entire semantic fabric of a document. We test the performance of several types
of pre-trained embeddings in neural LMs, and we investigate whether it is
possible to make the LM more aware of global semantic information through
embeddings pre-trained with a domain classification model. Initial experiments
suggest that as long as the proper objective criterion is used during training,
pre-trained embeddings are likely to be beneficial for neural language
modeling.

8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
22
The categories of this paper are: chao-dyn
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
9
The categories of this paper are: nlin
29
The categories of this paper are: funct-an
25
The categories of this paper are: patt-sol
34
The categories of this paper are: chem-ph
10
The categories of this paper are: q-bio
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
16
The categories of this paper are: stat