Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improving Language Generation with Sentence Coherence Objective   Conditional story generation and contextual text continuation have become
increasingly popular topics in NLP community. Existing models are often prone
to output paragraphs of texts that gradually diverge from the given prompt.
Although the generated text may have a reasonable perplexity and diversity, it
could easily be identified by human as gibberish. The goal of our project is to
improve the coherence and consistency across sentences in a language-generation
model. We aim to solve this issue by first training a sentence pair coherence
classifier with GPT-2 pretrained model, and then co-train the GPT-2 language
model with this new coherence objective using a method analogous to the
REINFORCE algorithm. This fine-tuned language model is able to generate lengthy
paragraph conditioned on a given topic without diverging too much. The
simplicity of this model allows it to be applicable to a variety of underlying
language model architecture since it only modifies the final layer of the
pre-trained model.

31
The categories of this paper are: ao-sci
35
The categories of this paper are: mtrl-th
29
The categories of this paper are: funct-an
14
The categories of this paper are: math-ph
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
7
The categories of this paper are: hep-th
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
26
The categories of this paper are: bayes-an
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
12
The categories of this paper are: nucl-th
27
The categories of this paper are: comp-gas
19
The categories of this paper are: eess