Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Consistency of a Recurrent Language Model With Respect to Incomplete
  Decoding   Despite strong performance on a variety of tasks, neural sequence models
trained with maximum likelihood have been shown to exhibit issues such as
length bias and degenerate repetition. We study the related issue of receiving
infinite-length sequences from a recurrent language model when using common
decoding algorithms. To analyze this issue, we first define inconsistency of a
decoding algorithm, meaning that the algorithm can yield an infinite-length
sequence that has zero probability under the model. We prove that commonly used
incomplete decoding algorithms - greedy search, beam search, top-k sampling,
and nucleus sampling - are inconsistent, despite the fact that recurrent
language models are trained to produce sequences of finite length. Based on
these insights, we propose two remedies which address inconsistency: consistent
variants of top-k and nucleus sampling, and a self-terminating recurrent
language model. Empirical results show that inconsistency occurs in practice,
and that the proposed methods prevent inconsistency.

18
The categories of this paper are: econ
2
The categories of this paper are: math
24
The categories of this paper are: dg-ga
20
The categories of this paper are: acc-phys
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
10
The categories of this paper are: q-bio
3
The categories of this paper are: physics
26
The categories of this paper are: bayes-an
11
The categories of this paper are: hep-lat
28
The categories of this paper are: alg-geom
30
The categories of this paper are: q-alg
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin