Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adaptive Input Representations for Neural Language Modeling   We introduce adaptive input representations for neural language modeling
which extend the adaptive softmax of Grave et al. (2017) to input
representations of variable capacity. There are several choices on how to
factorize the input and output layers, and whether to model words, characters
or sub-word units. We perform a systematic comparison of popular choices for a
self-attentional architecture. Our experiments show that models equipped with
adaptive embeddings are more than twice as fast to train than the popular
character input CNN while having a lower number of parameters. On the
WikiText-103 benchmark we achieve 18.7 perplexity, an improvement of 10.5
perplexity compared to the previously best published result and on the Billion
Word benchmark, we achieve 23.02 perplexity.

21
The categories of this paper are: adap-org
11
The categories of this paper are: hep-lat
9
The categories of this paper are: nlin
10
The categories of this paper are: q-bio
30
The categories of this paper are: q-alg
26
The categories of this paper are: bayes-an
33
The categories of this paper are: plasm-ph
20
The categories of this paper are: acc-phys
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
17
The categories of this paper are: q-fin
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
2
The categories of this paper are: math
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th