Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Corruption Is Not All Bad: Incorporating Discourse Structure into
  Pre-training via Corruption for Essay Scoring   Existing approaches for automated essay scoring and document representation
learning typically rely on discourse parsers to incorporate discourse structure
into text representation. However, the performance of parsers is not always
adequate, especially when they are used on noisy texts, such as student essays.
In this paper, we propose an unsupervised pre-training approach to capture
discourse structure of essays in terms of coherence and cohesion that does not
require any discourse parser or annotation. We introduce several types of
token, sentence and paragraph-level corruption techniques for our proposed
pre-training approach and augment masked language modeling pre-training with
our pre-training method to leverage both contextualized and discourse
information. Our proposed unsupervised approach achieves new state-of-the-art
result on essay Organization scoring task.

22
The categories of this paper are: chao-dyn
14
The categories of this paper are: math-ph
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
21
The categories of this paper are: adap-org
23
The categories of this paper are: solv-int
28
The categories of this paper are: alg-geom
10
The categories of this paper are: q-bio
5
The categories of this paper are: gr-qc
20
The categories of this paper are: acc-phys
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
24
The categories of this paper are: dg-ga
17
The categories of this paper are: q-fin
7
The categories of this paper are: hep-th