Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Implicit Length Bias of Label Smoothing on Beam Search Decoding   Label smoothing is ubiquitously applied in Neural Machine Translation (NMT)
training. While label smoothing offers a desired regularization effect during
model training, in this paper we demonstrate that it nevertheless introduces
length biases in the beam search decoding procedure. Our analysis shows that
label smoothing implicitly applies a length penalty term to output sequence,
causing a bias towards shorter translations. We also show that for a model
fully optimized with label smoothing, translation length is implicitly upper
bounded by a fixed constant independent of input. We verify our theory by
applying a simple rectification function at inference time to restore the
unbiased distributions from the label-smoothed model predictions. This
rectification method led to consistent quality improvements on WMT
English-German, English-French, English-Czech and English-Chinese tasks, up to
+0.3 BLEU at beam size 4 and +2.8 BLEU at beam size 200.

8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
20
The categories of this paper are: acc-phys
28
The categories of this paper are: alg-geom
10
The categories of this paper are: q-bio
6
The categories of this paper are: astro-ph
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
19
The categories of this paper are: eess