Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Self-Normalization Properties of Language Modeling   Self-normalizing discriminative models approximate the normalized probability
of a class without having to compute the partition function. In the context of
language modeling, this property is particularly appealing as it may
significantly reduce run-times due to large word vocabularies. In this study,
we provide a comprehensive investigation of language modeling
self-normalization. First, we theoretically analyze the inherent
self-normalization properties of Noise Contrastive Estimation (NCE) language
models. Then, we compare them empirically to softmax-based approaches, which
are self-normalized using explicit regularization, and suggest a hybrid model
with compelling properties. Finally, we uncover a surprising negative
correlation between self-normalization and perplexity across the board, as well
as some regularity in the observed errors, which may potentially be used for
improving self-normalization algorithms in the future.

8
The categories of this paper are: hep-ex
3
The categories of this paper are: physics
37
The categories of this paper are: supr-con
14
The categories of this paper are: math-ph
2
The categories of this paper are: math
18
The categories of this paper are: econ
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
19
The categories of this paper are: eess
36
The categories of this paper are: cmp-lg
20
The categories of this paper are: acc-phys
34
The categories of this paper are: chem-ph
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph