Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: How much do language models copy from their training data? Evaluating
  linguistic novelty in text generation using RAVEN   Current language models can generate high-quality text. Are they simply
copying text they have seen before, or have they learned generalizable
linguistic abstractions? To tease apart these possibilities, we introduce
RAVEN, a suite of analyses for assessing the novelty of generated text,
focusing on sequential structure (n-grams) and syntactic structure. We apply
these analyses to four neural language models (an LSTM, a Transformer,
Transformer-XL, and GPT-2). For local structure - e.g., individual dependencies
- model-generated text is substantially less novel than our baseline of
human-generated text from each model's test set. For larger-scale structure -
e.g., overall sentence structure - model-generated text is as novel or even
more novel than the human-generated baseline, but models still sometimes copy
substantially, in some cases duplicating passages over 1,000 words long from
the training set. We also perform extensive manual analysis showing that
GPT-2's novel text is usually well-formed morphologically and syntactically but
has reasonably frequent semantic issues (e.g., being self-contradictory).

25
The categories of this paper are: patt-sol
26
The categories of this paper are: bayes-an
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
7
The categories of this paper are: hep-th
14
The categories of this paper are: math-ph
11
The categories of this paper are: hep-lat
18
The categories of this paper are: econ
13
The categories of this paper are: quant-ph
15
The categories of this paper are: nucl-ex
2
The categories of this paper are: math
36
The categories of this paper are: cmp-lg
6
The categories of this paper are: astro-ph
0
The categories of this paper are: hep-ph
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin