Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Revisiting Summarization Evaluation for Scientific Articles   Evaluation of text summarization approaches have been mostly based on metrics
that measure similarities of system generated summaries with a set of human
written gold-standard summaries. The most widely used metric in summarization
evaluation has been the ROUGE family. ROUGE solely relies on lexical overlaps
between the terms and phrases in the sentences; therefore, in cases of
terminology variations and paraphrasing, ROUGE is not as effective. Scientific
article summarization is one such case that is different from general domain
summarization (e.g. newswire data). We provide an extensive analysis of ROUGE's
effectiveness as an evaluation metric for scientific summarization; we show
that, contrary to the common belief, ROUGE is not much reliable in evaluating
scientific summaries. We furthermore show how different variants of ROUGE
result in very different correlations with the manual Pyramid scores. Finally,
we propose an alternative metric for summarization evaluation which is based on
the content relevance between a system generated summary and the corresponding
human written summaries. We call our metric SERA (Summarization Evaluation by
Relevance Analysis). Unlike ROUGE, SERA consistently achieves high correlations
with manual scores which shows its effectiveness in evaluation of scientific
article summarization.

13
The categories of this paper are: quant-ph
8
The categories of this paper are: hep-ex
26
The categories of this paper are: bayes-an
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math
3
The categories of this paper are: physics
12
The categories of this paper are: nucl-th
22
The categories of this paper are: chao-dyn
4
The categories of this paper are: cond-mat
10
The categories of this paper are: q-bio
27
The categories of this paper are: comp-gas
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
28
The categories of this paper are: alg-geom
21
The categories of this paper are: adap-org
24
The categories of this paper are: dg-ga