Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Compression, Transduction, and Creation: A Unified Framework for
  Evaluating Natural Language Generation   Natural language generation (NLG) spans a broad range of tasks, each of which
serves for specific objectives and desires different properties of generated
text. The complexity makes automatic evaluation of NLG particularly
challenging. Previous work has typically focused on a single task and developed
individual evaluation metrics based on specific intuitions. In this paper, we
propose a unifying perspective that facilitates the design of metrics for a
wide range of language generation tasks and quality aspects. Based on the
nature of information change from input to output, we classify NLG tasks into
compression (e.g., summarization), transduction (e.g., text rewriting), and
creation (e.g., dialog). The information alignment, or overlap, between input,
context, and output text plays a common central role in characterizing the
generation. Using the uniform concept of information alignment, we develop a
family of interpretable metrics for various NLG tasks and aspects, often
without need of gold reference data. To operationalize the metrics, we train
self-supervised models to approximate information alignment as a prediction
task. Experiments show the uniformly designed metrics achieve stronger or
comparable correlations with human judgement compared to state-of-the-art
metrics in each of diverse tasks, including text summarization, style transfer,
and knowledge-grounded dialog. With information alignment as the intermediate
representation, we deliver a composable library for easy NLG evaluation and
future metric design.

19
The categories of this paper are: eess
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
30
The categories of this paper are: q-alg
25
The categories of this paper are: patt-sol
36
The categories of this paper are: cmp-lg
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con