Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction
  from Language Models   Coherent discourse is distinguished from a mere collection of utterances by
the satisfaction of a diverse set of constraints, for example choice of
expression, logical relation between denoted events, and implicit compatibility
with world-knowledge. Do neural language models encode such constraints? We
design an extendable set of test suites addressing different aspects of
discourse and dialogue coherence. Unlike most previous coherence evaluation
studies, we address specific linguistic devices beyond sentence order
perturbations, allowing for a more fine-grained analysis of what constitutes
coherence and what neural models trained on a language modelling objective do
encode. Extending the targeted evaluation paradigm for neural language models
(Marvin and Linzen, 2018) to phenomena beyond syntax, we show that this
paradigm is equally suited to evaluate linguistic qualities that contribute to
the notion of coherence.

5
The categories of this paper are: gr-qc
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th
11
The categories of this paper are: hep-lat
36
The categories of this paper are: cmp-lg
25
The categories of this paper are: patt-sol
7
The categories of this paper are: hep-th
33
The categories of this paper are: plasm-ph
31
The categories of this paper are: ao-sci
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int