Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Analysis of GraphSum's Attention Weights to Improve the Explainability
  of Multi-Document Summarization   Modern multi-document summarization (MDS) methods are based on transformer
architectures. They generate state of the art summaries, but lack
explainability. We focus on graph-based transformer models for MDS as they
gained recent popularity. We aim to improve the explainability of the
graph-based MDS by analyzing their attention weights. In a graph-based MDS such
as GraphSum, vertices represent the textual units, while the edges form some
similarity graph over the units. We compare GraphSum's performance utilizing
different textual units, i. e., sentences versus paragraphs, on two news
benchmark datasets, namely WikiSum and MultiNews. Our experiments show that
paragraph-level representations provide the best summarization performance.
Thus, we subsequently focus oAnalysisn analyzing the paragraph-level attention
weights of GraphSum's multi-heads and decoding layers in order to improve the
explainability of a transformer-based MDS model. As a reference metric, we
calculate the ROUGE scores between the input paragraphs and each sentence in
the generated summary, which indicate source origin information via text
similarity. We observe a high correlation between the attention weights and
this reference metric, especially on the the later decoding layers of the
transformer architecture. Finally, we investigate if the generated summaries
follow a pattern of positional bias by extracting which paragraph provided the
most information for each generated summary. Our results show that there is a
high correlation between the position in the summary and the source origin.

19
The categories of this paper are: eess
34
The categories of this paper are: chem-ph
31
The categories of this paper are: ao-sci
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom
36
The categories of this paper are: cmp-lg
10
The categories of this paper are: q-bio
2
The categories of this paper are: math
26
The categories of this paper are: bayes-an
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
22
The categories of this paper are: chao-dyn
11
The categories of this paper are: hep-lat
16
The categories of this paper are: stat