Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Towards an Automatic Turing Test: Learning to Evaluate Dialogue
  Responses   Automatically evaluating the quality of dialogue responses for unstructured
domains is a challenging problem. Unfortunately, existing automatic evaluation
metrics are biased and correlate very poorly with human judgements of response
quality. Yet having an accurate automatic evaluation procedure is crucial for
dialogue research, as it allows rapid prototyping and testing of new models
with fewer expensive human evaluations. In response to this challenge, we
formulate automatic dialogue evaluation as a learning problem. We present an
evaluation model (ADEM) that learns to predict human-like scores to input
responses, using a new dataset of human response scores. We show that the ADEM
model's predictions correlate significantly, and at a level much higher than
word-overlap metrics such as BLEU, with human judgements at both the utterance
and system-level. We also show that ADEM can generalize to evaluating dialogue
models unseen during training, an important step for automatic dialogue
evaluation.

14
The categories of this paper are: math-ph
31
The categories of this paper are: ao-sci
18
The categories of this paper are: econ
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
6
The categories of this paper are: astro-ph
21
The categories of this paper are: adap-org
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
23
The categories of this paper are: solv-int
29
The categories of this paper are: funct-an
15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
5
The categories of this paper are: gr-qc
20
The categories of this paper are: acc-phys