Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness   Large-scale dialogue datasets have recently become available for training
neural dialogue agents. However, these datasets have been reported to contain a
non-negligible number of unacceptable utterance pairs. In this paper, we
propose a method for scoring the quality of utterance pairs in terms of their
connectivity and relatedness. The proposed scoring method is designed based on
findings widely shared in the dialogue and linguistics research communities. We
demonstrate that it has a relatively good correlation with the human judgment
of dialogue quality. Furthermore, the method is applied to filter out
potentially unacceptable utterance pairs from a large-scale noisy dialogue
corpus to ensure its quality. We experimentally confirm that training data
filtered by the proposed method improves the quality of neural dialogue agents
in response generation.

17
The categories of this paper are: q-fin
35
The categories of this paper are: mtrl-th
20
The categories of this paper are: acc-phys
0
The categories of this paper are: hep-ph
34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
15
The categories of this paper are: nucl-ex
11
The categories of this paper are: hep-lat
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
36
The categories of this paper are: cmp-lg
22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
37
The categories of this paper are: supr-con