Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Change that Matters in Discourse Parsing: Estimating the Impact of
  Domain Shift on Parser Error   Discourse analysis allows us to attain inferences of a text document that
extend beyond the sentence-level. The current performance of discourse models
is very low on texts outside of the training distribution's coverage,
diminishing the practical utility of existing models. There is need for a
measure that can inform us to what extent our model generalizes from the
training to the test sample when these samples may be drawn from distinct
distributions. While this can be estimated via distribution shift, we argue
that this does not directly correlate with change in the observed error of a
classifier (i.e. error-gap). Thus, we propose to use a statistic from the
theoretical domain adaptation literature which can be directly tied to
error-gap. We study the bias of this statistic as an estimator of error-gap
both theoretically and through a large-scale empirical study of over 2400
experiments on 6 discourse datasets from domains including, but not limited to:
news, biomedical texts, TED talks, Reddit posts, and fiction. Our results not
only motivate our proposal and help us to understand its limitations, but also
provide insight on the properties of discourse models and datasets which
improve performance in domain adaptation. For instance, we find that non-news
datasets are slightly easier to transfer to than news datasets when the
training and test sets are very different. Our code and an associated Python
package are available to allow practitioners to make more informed model and
dataset choices.

24
The categories of this paper are: dg-ga
20
The categories of this paper are: acc-phys
10
The categories of this paper are: q-bio
37
The categories of this paper are: supr-con
2
The categories of this paper are: math
35
The categories of this paper are: mtrl-th
21
The categories of this paper are: adap-org
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat
8
The categories of this paper are: hep-ex