Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multi-Task Self-Supervised Learning for Disfluency Detection   Most existing approaches to disfluency detection heavily rely on
human-annotated data, which is expensive to obtain in practice. To tackle the
training data bottleneck, we investigate methods for combining multiple
self-supervised tasks-i.e., supervised tasks where data can be collected
without manual labeling. First, we construct large-scale pseudo training data
by randomly adding or deleting words from unlabeled news data, and propose two
self-supervised pre-training tasks: (i) tagging task to detect the added noisy
words. (ii) sentence classification to distinguish original sentences from
grammatically-incorrect sentences. We then combine these two tasks to jointly
train a network. The pre-trained network is then fine-tuned using
human-annotated disfluency detection training data. Experimental results on the
commonly used English Switchboard test set show that our approach can achieve
competitive performance compared to the previous systems (trained using the
full dataset) by using less than 1% (1000 sentences) of the training data. Our
method trained on the full dataset significantly outperforms previous methods,
reducing the error by 21% on English Switchboard.

17
The categories of this paper are: q-fin
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
7
The categories of this paper are: hep-th
37
The categories of this paper are: supr-con
5
The categories of this paper are: gr-qc
29
The categories of this paper are: funct-an
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
11
The categories of this paper are: hep-lat
10
The categories of this paper are: q-bio
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org