Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question
  Answering   Disfluencies is an under-studied topic in NLP, even though it is ubiquitous
in human conversation. This is largely due to the lack of datasets containing
disfluencies. In this paper, we present a new challenge question answering
dataset, Disfl-QA, a derivative of SQuAD, where humans introduce contextual
disfluencies in previously fluent questions. Disfl-QA contains a variety of
challenging disfluencies that require a more comprehensive understanding of the
text than what was necessary in prior datasets. Experiments show that the
performance of existing state-of-the-art question answering models degrades
significantly when tested on Disfl-QA in a zero-shot setting.We show data
augmentation methods partially recover the loss in performance and also
demonstrate the efficacy of using gold data for fine-tuning. We argue that we
need large-scale disfluency datasets in order for NLP models to be robust to
them. The dataset is publicly available at:
https://github.com/google-research-datasets/disfl-qa.

33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
27
The categories of this paper are: comp-gas
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
15
The categories of this paper are: nucl-ex
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
8
The categories of this paper are: hep-ex
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
20
The categories of this paper are: acc-phys