Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework
  for Natural Language Generation   Current pre-training works in natural language generation pay little
attention to the problem of exposure bias on downstream tasks. To address this
issue, we propose an enhanced multi-flow sequence to sequence pre-training and
fine-tuning framework named ERNIE-GEN, which bridges the discrepancy between
training and inference with an infilling generation mechanism and a noise-aware
generation method. To make generation closer to human writing patterns, this
framework introduces a span-by-span generation flow that trains the model to
predict semantically-complete spans consecutively rather than predicting word
by word. Unlike existing pre-training methods, ERNIE-GEN incorporates
multi-granularity target sampling to construct pre-training data, which
enhances the correlation between encoder and decoder. Experimental results
demonstrate that ERNIE-GEN achieves state-of-the-art results with a much
smaller amount of pre-training data and parameters on a range of language
generation tasks, including abstractive summarization (Gigaword and
CNN/DailyMail), question generation (SQuAD), dialogue generation (Persona-Chat)
and generative question answering (CoQA).

29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
5
The categories of this paper are: gr-qc
8
The categories of this paper are: hep-ex
6
The categories of this paper are: astro-ph
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
22
The categories of this paper are: chao-dyn
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat
3
The categories of this paper are: physics
18
The categories of this paper are: econ
10
The categories of this paper are: q-bio
0
The categories of this paper are: hep-ph
27
The categories of this paper are: comp-gas
28
The categories of this paper are: alg-geom