\section{Introduction}\label{sec:intro}
Social media posts with false claims have led to real-world threats on many aspects such as politics~\citep{pizzagate}, social order~\citep{salt}, and personal health~\citep{shuanghuanglian}.
To tackle this issue, over 300 fact-checking projects have been launched, such as Snopes\footnote{\url{https://www.snopes.com}} and Jiaozhen\footnote{\url{https://fact.qq.com/}}~\citep{duke-report}. Meanwhile, automatic systems have been developed for detecting suspicious claims on social media~\citep{newsverify, credeye}.
This is however not the end. A considerable amount of false claims continually spread, even though they are already proved false.
According to a recent report~\citep{tencent-report}, around 12\% of false claims published on Chinese social media, are actually ``old'', as they have been debunked previously. Hence, detecting previously fact-checked claims is an important task.

According to the seminal work by \citet{shaar2020}, the task is tackled by a two-stage information retrieval approach. Its typical workflow is illustrated in  Figure~\ref{fig1}(a).
Given a claim as a query, in the first stage a basic searcher (e.g., BM25 \citealp{bm25}) searches for candidate articles from a collection of fact-checking articles (FC-articles). In the second stage, a more powerful model (e.g.,  BERT,~\citealp{bert}) reranks the candidates to provide evidence for manual or automatic detection.
Existing works focus on the reranking stage: \citet{vo2020} model the interactions between a claim and the whole candidate articles, while \citet{shaar2020} extract several semantically similar sentences from FC-articles as a proxy.
Nevertheless, these methods treat FC-articles as \emph{general} documents and ignore characteristics of FC-articles. Figure~\ref{fig1}(b) shows three sentences from candidate articles for the given claim. Among them, S1 is more friendly to semantic matching than S2 and S3 because the whole S1 focuses on describing its topic and does not contain tokens irrelevant to the given claim, e.g., "has spread over years" in S2. Thus, a semantic-based model does not require to have strong filtering capability. If we use only general methods on this task, the relevant S2 and S3 may be neglected while irrelevant S1 is focused. To let the model focus on key sentences (i.e., sentences as a good proxy of article-level relevance) like S2 and S3, we need to consider two characteristics of FC-articles besides semantics: \textbf{C1}. Claims are often quoted to describe the checked events (e.g., the underlined text in S2); \textbf{C2}. Event-irrelevant patterns to introduce or debunk claims are common in FC-articles (e.g., bold texts in S2 and S3).

Based on the observations, we propose a novel reranker, \model~(\underline{M}emory-enhanced \underline{T}ransformers for \underline{M}atching). The reranker identifies key sentences per article using claim- and pattern-sentence relevance, and then integrates information from the claim, key sentences, and patterns for article-level relevance prediction. In particular, regarding \textbf{C1}, we propose \texttt{ROUGE}-guided Transformer (ROT) to score claim-sentence relevance literally and semantically.

As for \textbf{C2}, we obtain the pattern vectors by clustering the difference of sentence and claim vectors for scoring pattern-sentence relevance and store them in the Pattern Memory Bank (PMB).
The joint use of ROT and PMB allows us to identify key sentences that reflect the two characteristics of FC-articles.
Subsequently, fine-grained interactions among claims and key sentences are modeled by the multi-layer Transformer and aggregated with patterns to obtain an article-level feature representation. The article feature is fed into a Multi-layer Perceptron (MLP) to predict the claim-article relevance.

To validate the effectiveness of our method, we built the first Chinese dataset for this task with 11,934 claims collected from Chinese Weibo\footnote{\url{https://weibo.com}} and 27,505 fact-checking articles from multiple sources. 39,178 claim-article pairs are annotated as relevant. Experiments on the English dataset and the newly built Chinese dataset show that \model~outperforms existing methods. Further human evaluation and case studies prove that \model~ finds key sentences as explanations. Our main contributions are as follows:
\begin{compactitem}
	\item We propose a novel reranker \model~for fact-checked claim detection, which can better identify key sentences in fact-checking articles by exploiting their characteristics.
	\item We design \texttt{ROUGE}-guided Transformer to combine lexical and semantic information and propose a memory mechanism to capture and exploit common patterns in fact-checking articles.
	\item Experiments on two real-world datasets show that \model~outperforms existing methods. Further human evaluation and case studies prove that our model finds key sentences as good explanations.
	\item We built the first Chinese dataset for fact-checked claim detection with fact-checking articles from diverse sources.
\end{compactitem}

\begin{figure*}[ht]
	\centering
	\includegraphics[width=\textwidth]{figs/arch2_cr.pdf}
	\caption{Architecture of \model. Given a claim $q$ and a candidate article $d$ with $l$ sentences, $s_1,\mydots,s_l$, \model~\ding{172} feeds $(q,s)$ pairs into \texttt{ROUGE}-guided Transformer (ROT) to obtain claim-sentence scores in both lexical and semantic aspects; \ding{173} matches residual embeddings $\bm{r}_{s,q}$ with vectors in Pattern Memory Bank (PMB) (here, only four are shown) to obtain pattern-sentence scores; \ding{174} identifies $k_2$ key sentences by combining the two scores (here, $k_2=2$, and $s_i$ and $s_l$ are selected); \ding{175} models interaction among $\bm{q^\prime},\bm{s^\prime},$ and the nearest memory vector $\bm{m}$ for each key sentence; and \ding{176} perform score-weighted aggregation and predict the claim-article relevance.}
	\label{fig:arch}
\end{figure*}

\section{Related Work}\label{sec:rw}
To defend against false information, researchers are mainly devoted to two threads: (1) \textbf{Automatic fact-checking} methods mainly retrieve relevant factual information from designated sources and judge the claim's veracity. \citet{fever} use Wikipedia as a fact tank and build a shared task for automatic fact-checking, while \citet{declare} and \citet{RDD} retrieve webpages as evidence and use their stances on claims for veracity prediction. (2) \textbf{Fake news detection} methods often use non-factual signals, such as styles~\citep{style-aaai,style-image}, emotions~\citep{ajao, dual-emotion}, source credibility~\citep{fang}, user response~\citep{defend} and diffusion network~\citep{early, alone}. However, these methods mainly aim at newly emerged claims and do not address those claims that have been fact-checked but continually spread. Our work is in a new thread, \textbf{detecting previously fact-checked claims}. \citet{vo2020} models interaction between claims and FC-articles by combining GloVe~\citep{glove} and ELMo embeddings~\citep{elmo}. \citet{shaar2020} train a RankSVM with scores from BM25 and Sentence-BERT for relevance prediction. These methods ignore the characteristics of FC-articles, which limits the ranking performance and explainability.

\section{Proposed Method}\label{sec:method}

Given a claim $q$ and a candidate set of $k_1$ FC-articles $\mathcal{D}$ obtained by a standard full-text retrieval model (BM25), we aim to rerank FC-articles truly relevant w.r.t. $q$ at the top by modeling fine-grained relevance between $q$ and each article $d\in \mathcal{D}$. This is accomplished by Memory-enhanced Transformers for Matching (\model), which conceptually has two steps, (1) Key Sentence Identification and (2) Article Relevance Prediction, see Figure~\ref{fig:arch}.
For an article of $l$ sentences, let $\mathcal{S}=\{s_{1},\mydots,s_{l}\}$ be its sentence set.
In Step (1), for each sentence, we derive claim-sentence relevance score from \texttt{ROUGE}-guided Transformer (ROT) and pattern-sentence relevance score from Pattern Memory Bank (PMB). The scores indicate how similar the sentence is to the claim and pattern vectors, i.e., how possible to be a key sentence. Top $k_2$ sentences are selected for more complicated interactions and aggregation with the claim and pattern vectors in Step (2). The aggregated vector is used for the final prediction. We detail the components and then summarize the training procedure below.

\subsection{Key Sentence Identification}
\subsubsection{\texttt{ROUGE}-guided Transformer (ROT)}

ROT (left top of Figure.~\ref{fig:arch}) is used to evaluate the relevance between $q$ and each sentence $s$ in $\{\mathcal{S}_i\}^{k_1}_{i=1}$, both lexically and semantically.
Inspired by \citep{complementing}, we choose to ``inject'' the ability to consider lexical relevance into the semantic model. As the BERT is proved to capture and evaluate semantic relevance~\citep{BERTScore}, we use a one-layer Transformer initialized with the first block of pretrained BERT to obtain the initial semantic representation of $q$ and $s$:
\begin{equation}
	{\bm z}_{q,s} = \mathrm{Transformer}\left(\mathtt{[CLS]}\ q\ \mathtt{[SEP]}\ s \right)
\end{equation}
where $\mathtt{[CLS]}$ and $\mathtt{[SEP]}$ are preserved tokens and $\bm{z}_{q,s}$ is the output representation.

To force ROT to consider the lexical relevance, we finetune the pretrained Transformer with the guidance of \texttt{ROUGE}~\citep{rouge}, a widely-used metric to evaluate the lexical similarity of two segments in summarization and translation tasks. The intuition is that lexical relevance can be characterized by token overlapping, which ROUGE exactly measures. We minimize the mean square error between the prediction and the precision and recall of \texttt{ROUGE-2} between $q$ and $s$ ($\texttt{R}_{2} \in \mathbb{R}^2$) to optimize the ROT:
\begin{equation}
    \hat{\texttt{R}}(q,s)=\mathrm{MLP}\big(\bm{z}_{q,s}(\mathtt{[CLS]})\big)
\end{equation}
\begin{equation}
    \mathcal{L}_{R} = \Vert\hat{\texttt{R}}(q,s)-\texttt{R}_{2}(q,s)\Vert^2_2 + \lambda_R\Vert\Delta\theta\Vert^2_2
\end{equation}
where the first term is the regression loss and the second is to constraint the change of parameters as the ability to capture semantic relevance should be maintained. $\lambda_R$ is a control factor and $\Delta\theta$ represents the change of parameters.

\subsubsection{Pattern Memory Bank (PMB)}
The Pattern Memory Bank (PMB) is to generate, store, and update the vectors which represent the common patterns in FC-articles. The vectors in PMB will be used to evaluate pattern-sentence relevance (see~Section \ref{sec:selection}). Here we detail how to formulate, initialize, and update these patterns below.

\noindent\textbf{Formulation.}
Intuitively, one can summarize the templates, like ``\mydots has been debunked by\mydots'', and explicitly do \emph{exact} matching, but the templates are costly to obtain and hard to integrate into neural models.
Instead, we \emph{implicitly} represent the common patterns using vectors derived from embeddings of our model, ROT.
Inspired by~\citep{memory-bank}, we use a memory bank $\mathcal{M}$ to store $K$ common patterns (as vectors), i.e., $\mathcal{M}=\{\bm{m}_i\}^{K}_{i=1}$.

\noindent\textbf{Initialization.} We first represent each $q$ in the training set and $s$ in the corresponding articles by averaging its token embeddings (from the embedding layer of ROT).
Considering that a pattern vector should be \emph {event-irrelevant}, we heuristically remove the event-related part in $s$ as possible by calculating the residual embeddings $\bm{r}_{s,q}$, i.e., subtracting $\bm{q}$ from $\bm{s}$.
We rule out the residual embeddings that do not satisfy $t_{low}<\left\|\bm{r}_{s,q}\right\|_2<t_{high}$, because they are unlikely to contain good pattern information: $\left\|\bm{r}_{s,q}\right\|_2\leq t_{low}$ indicates $q$ and $s$ are highly similar and thus leave little pattern information, while $\left\|\bm{r}_{s,q}\right\|_2\geq t_{high}$ indicates $s$ may not align with $q$ in terms of the event, so the corresponding $r_{s,q}$ is of little sense.
Finally, we aggregate the valid residual embeddings into $K$ clusters using K-means and obtain the initial memory bank $\mathcal{M}$:
\begin{equation}
	\mathcal{M}=\mathrm{K}\mbox{-}\mathrm{means}\big(\{\bm{r}_{s,q}^{valid}\}\big) \!=\! \{\bm{m}_1,\mydots,\bm{m}_K\}
\end{equation}
where $\{\bm{r}_{s,q}^{valid}\}$ is the set of valid residual embeddings.

\noindent\textbf{Update.} As the initial $K$ vectors may not accurately represent common patterns, we update the memory bank according to the feedbacks of results during training: If the model predicts rightly, the key sentence, say $s$, should be used to update its nearest pattern vector $m$. To maintain stability, we use an epoch-wise update instead of an iteration-wise update.

Take updating $\bm{m}$ as an example. After an epoch, we extract all $n$ key sentences whose nearest pattern vector is $\bm{m}$ and their $n$ corresponding claims, which is denoted as a tuple set $(\mathcal{S}, \mathcal{Q})^m$. Then $(\mathcal{S}, \mathcal{Q})^m$ is separated into two subsets, $\mathcal{R}^{m}$ and $\mathcal{W}^{m}$, which contain $n_r$ and $n_w$ sentence-claim tuples from the rightly and wrongly predicted samples, respectively. The core of our update mechanism (Figure~\ref{fig:mem_update}) is to draw $\bm{m}$ closer to the residual embeddings in $\mathcal{R}^{m}$ and push it away from those in $\mathcal{W}^{m}$. We denote the $i^{th}$ residual embedding from the two subsets as $\bm{r}_{\scriptscriptstyle \mathcal{R}_i^m}$ and $\bm{r}_{\scriptscriptstyle \mathcal{W}_i^m}$, respectively.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.43\textwidth]{figs/mem_update}
	\caption{Illustration for Memory Vector Update.}
	\label{fig:mem_update}
\end{figure}

To determine the update direction, we calculate a weighted sum of residual embeddings according to the predicted matching scores.
For $(s,q)$, suppose \model~output $\hat{y}_{s,q} \in [0,1]$ as the predicted matching score of $q$ and $d$ (whose key sentence is $s$), the weight of $\bm{r}_{s,q}$ is $|\hat{y}_{s,q}-0.5|$ (denoted as $w_{s,q}$). Weighted residual embeddings are respectively summed and normalized as the components of the direction vector (Eq.~\eqref{eq:aggregated_res}):
\begin{equation}
\label{eq:aggregated_res}
	\bm{u}^{mr}\!= \!\bigg( \sum_{i=1}^{n_r}  w_{\scriptscriptstyle \mathcal{R}_i^m}\bm{r}_{\scriptscriptstyle \mathcal{R}_i^m} \bigg),
	\bm{u}^{mw}\!=\!\bigg( \sum_{i=1}^{n_w}  w_{\scriptscriptstyle \mathcal{W}_i^m} \bm{r}_{\scriptscriptstyle \mathcal{W}_i^m}\bigg)
\end{equation}
where $\bm{u}^{mr}$ and $\bm{u}^{mw}$ are the aggregated residual embeddings. The direction is determined by Eq.~\eqref{eq:direction}:
\begin{equation}\label{eq:direction}
	\bm{u}^{m}=w_r\underbrace{(\bm{u}^{mr}-\bm{m})}_{\mbox{\small{draw closer}}}+ w_w\underbrace{(\bm{m}-\bm{u}^{mw})}_{\mbox{\small{push away}}}
\end{equation}
where $w_r$ and $w_w$ are the normalized sum of corresponding weights used in Eq.~\eqref{eq:aggregated_res} ($w_r+w_w=1$).
The pattern vector $m$ is updated with:
\begin{equation}
\label{eq:memory-update}
	\bm{m}_{new} = \bm{m}_{old} + \lambda_m \Vert\bm{m}_{old}\Vert _2 \frac{\bm{u}^{m}}{\Vert\bm{u}^{m}\Vert_2}
\end{equation}
where $\bm{m}_{old}$ and $\bm{m}_{new}$ are the memory vector $\bm{m}$ before and after updating; the constant $\lambda_m$ and $\left\|\bm{m}_{old}\right\|_2$ jointly control the step size.

\subsubsection{Key Sentence Selection}\label{sec:selection}
Whether a sentence is selected as a key sentence is determined by combining claim- and pattern-sentence relevance scores. The former is calculated with the distance of $q$ and $s$ trained with ROT (Eq.~\eqref{eq:score_q}) and the latter uses the distance between the nearest pattern vector in PMB and the residual embedding (Eq.~\eqref{eq:score_p}). The scores are scaled to $[0,1]$. For each sentence $s$ in $d$, the relevance score with $q$ is calculated by Eq.~\eqref{eq:score_all}:
\begin{equation}
	scr_Q(q,s)=\mathrm{Scale}(\left\|\bm{r}_{s,q}\right\|_2)\\ \label{eq:score_q}
\end{equation}
\begin{equation}\label{eq:score_p}
	scr_P(q,s)=\mathrm{Scale}(\left\|\bm{m}_{u}-\bm{r}_{s,q}\right\|_2)
\end{equation}
\begin{equation}\label{eq:score_all}
	scr(q,s)=\lambda_{Q}scr_Q(q,s)+\lambda_{P}scr_P(q,s)
\end{equation}
where $\mathrm{Scale}(x)\!=\!1-\frac{x-min}{max-min}$ and $max$ and $min$ are the maximum and minimum distance of $s$ in $d$, respectively. $u=\arg \min_{i} \left\|\bm{m}_i-\bm{r}_{s,q}\right\|_2$, and $\lambda_{Q}$ and $\lambda_{P}$ are hyperparameters whose sum is $1$.

Finally, sentences with top-$k_2$ scores, denoted as $\mathcal{K}\! =\!\{s^{key}_i(q,d)\}^{k_2}_{i=1}$, are selected as the \emph{key sentences} in $d$ for the claim $q$.

\subsection{Article Relevance Prediction (ARP)}\label{sec:reasoner}
\noindent\textbf{Sentence representation.} We model more complicated interactions between the claim and the key sentences by feeding each $z_{q,s^{key}}$ (derived from ROT) into a multi-layer Transformer ($\mathrm{MultiTransformer}$):
\begin{equation}
\label{eq:reasoner1}
	\bm{z}^\prime_{q,s^{key}}= \mathrm{MultiTransformer}(\bm{z}_{q,s^{key}})
\end{equation}
Following~\citep{sentence-bert}, we respectively compute the mean of all output token vectors of $q$ and $s$ in $z^\prime_{q,s^{key}}$ to obtain the fixed sized sentence vectors $\bm{q}^\prime \in \mathbb{R}^{dim}$ and $\bm{s}^{key\prime} \in \mathbb{R}^{dim}$, where $dim$ is the dimension of a token in Transformers.

\noindent\textbf{Weighted memory-aware aggregation.} For final prediction, we use a score-weighted memory-aware aggregation. To make the predictor aware of the pattern information, we append the corresponding nearest pattern vectors to the claim and key sentence vectors:
\begin{equation}
\label{eq:qsv-concat}
	\bm{v}_i = [\bm{q}^\prime,\bm{s}^{key\prime}_i(q,d),\bm{m}_{j}]
\end{equation}
where $i\!=\!1,\mydots,k_2$. $j\!=\!\mathop{\arg\min}_{k} \left\|\bm{m}_k\!-\!\bm{r}_{s^{key}_i,q}\right\|_2$.

Intuitively, a sentence with higher score should be attended more. Thus, the concatenated vectors (Eq.~\eqref{eq:qsv-concat}) are weighted by the relevance scores from Eq.~\eqref{eq:score_all} (normalized across the top-$k_2$ sentences). The weighted aggregating vector is fed into a MLP which outputs the probability that $d$ fact-checks $q$:
\begin{equation}
	scr^\prime(q,s_i^{key}) = \mathrm{Normalize}\big(scr(q,s_i^{key})\big)
\end{equation}
\begin{equation}\label{eq:final-prediction}
	\hat{y}_{q,d}=\mathrm{MLP}\Big(\sum_{i=1}^{k_2}scr^\prime(q,s_i^{key}) \bm{v}_i\Big)
\end{equation}
where $\hat{y}_{q,d} \in [0,1]$. If $\hat{y}_{q,d}>0.5$, the model predicts that $d$ fact-checks $q$, otherwise does not. The loss function is cross entropy:
\begin{equation}\label{eq:celoss}
	\mathcal{L}_{M}=\mathrm{CrossEntropy}(\hat{y}_{q,d},y_{q,d})
\end{equation}
where $y_{q,d} \in \{0,1\}$ is the ground truth label. $y_{q,d}=1$ if $d$ fact-checks $q$ and $0$ otherwise. The predicted values are used to rank all $k_1$ candidate articles retrieved in the first stage.

\subsection{Training \model}
We summarize the training procedure of \model~in Algorithm~\ref{algo}, including the pretraining of ROT, the initialization of PMB, the training of ARP, and the epoch-wise update of PMB.

\begin{algorithm}[ht]
\caption{\model~Training Procedure}
\label{algo}
\begin{algorithmic}[1]
\Require
Training set {$\mathcal{T}=[(q_0,d_{00}),\mydots,(q_0,d_{0k_1}),$ $\mydots,(q_n,d_{nk_1})]$ where the $k_1$ candidate articles for each claim are retrieved by BM25.}
\State Pre-train \texttt{ROUGE}-guided Transformer.
\State Initialize the Pattern Memory Bank (PMB).
\For {each epoch}
	\For {$(q,d)$ in $\mathcal{T}$}
		\State {\color{gray} {//} Key Sentence Identification}
		\State Calculate $scr_Q(q,s)$ via ROT and $scr_P(q,s)$ via PMB.
		\State Calculate $scr(q,s)$ using Eq.~\eqref{eq:score_all}.
		\State Select key sentences $\mathcal{K}$.
		\State {\color{gray} {//} Article Relevance Prediction (ARP)}
		\State Calculate $\bm{v}$ for each $s$ in $\mathcal{K}$ and $\hat{y}_{q,d}$.
		\State Update the ARP to minimize $\mathcal{L}_M$.
	\EndFor
	\State Update the PMB using Eq.~\eqref{eq:memory-update}.
\EndFor
\end{algorithmic}
\end{algorithm}

\section{Experiments}\label{sec:exp}
In this section, we mainly answer the following experimental questions:

\noindent\textbf{EQ1:} Can \model~improve the ranking performance of FC-articles given a claim?

\noindent\textbf{EQ2:} How effective are the components of \model, including \texttt{ROUGE}-guided Transformer, Pattern Memory Bank, and weighted memory-aware aggregation in Article Relevance Prediction?

\noindent\textbf{EQ3:} To what extent can \model~identify key sentences in the articles, especially in the longer ones?

\subsection{Data}\label{sec:data}
We conducted the experiments on two real-world datasets. Table~\ref{tab:dataset} shows the statistics of the two datasets. The details are as follows:

\noindent\textbf{Twitter Dataset}

The Twitter\footnote{\url{https://twitter.com}} dataset is originated from \citep{vo2019} and processed by \citet{vo2020}.
The dataset pairs the claims (tweets) with the corresponding FC-articles from Snopes. For tweets with images, it appends the OCR results to the tweets.
We remove the manually normalized claims in Snopes' FC-articles to adapt to more general scenarios. The data split is the same as that in \citep{vo2020}.

\noindent\textbf{Weibo Dataset}

We built the first Chinese dataset for the task of detecting previously fact-checked claims in this article. The claims are collected from Weibo and the FC-articles are from \textit{multiple fact-checking sources} including Jiaozhen, Zhuoyaoji\footnote{\url{https://piyao.sina.cn}}, etc. We recruited annotators to match claims and FC-articles based on basic search results. Appendix~\ref{apd:dataset} introduce the details.

\begin{table}[t]
\caption{Statistics of the Twitter and the Weibo dataset. \#: Number of. C-A Pairs: Claim-article pairs.}
\label{tab:dataset}
{\small
\setlength\tabcolsep{3.2pt}
\begin{tabular}{l|rrr|rrr}
\hline
\multirow{2}{*}{\textbf{Dataset}} & \multicolumn{3}{c|}{\textbf{Twitter}} & \multicolumn{3}{c}{\textbf{Weibo}} \\ \cline{2-7}
 & Train & Val & Test & Train & Val & Test \\ \hline
\#Claim &  8,002 & 1,000 & 1,001 & 8,356 & 1,192 & 2,386\\
\#Articles  & 1,703  & 1,697 & 1,697 & 17,385 & 8,353 & 11,715\\
C-A Pairs  &  8,025  & 1,002 & 1,005 & 28,596 & 3,337 &  7,245\\ \hline
\multicolumn{7}{c}{\textbf{Relevant Fact-checking Articles Per Claim}} \\ \hline
Average & 1.003 & 1.002 & 1.004 & 3.422 & 2.799 & 3.036 \\
Medium & 1 & 1 & 1 & 2 & 1  & 2 \\
Maximum & 2 & 2 & 2 & 50 & 18 & 32 \\ \hline
\end{tabular}
}
\end{table}

\begin{table*}[htbp]
\centering
\caption{Performance of baselines and \model. Best results are in \textbf{boldface}.}
\label{tab:main_exp}
\setlength\tabcolsep{3.5pt}
{\small
\begin{tabular}{l|c|cccccc|cccccc}
\hline
\multirow{3}{*}{\textbf{Method}} & \multirow{3}{*}{\begin{tabular}[c]{@{}c@{}}\textbf{Selecting}\\ \textbf{Sentences?}\end{tabular}} & \multicolumn{6}{c|}{\textbf{Weibo}} & \multicolumn{6}{c}{\textbf{Twitter}} \\ \cline{3-14}
 & & \multirow{2}{*}{MRR} & \multicolumn{3}{c}{MAP@} & \multicolumn{2}{c|}{HIT@} & \multirow{2}{*}{MRR} & \multicolumn{3}{c}{MAP@} & \multicolumn{2}{c}{HIT@} \\ \cline{4-8} \cline{10-14}
 & & & 1 & 3 & 5 & 3 & 5 &  & 1 & 3 & 5 & 3 & 5\\ \hline
BM25 & & 0.709 & 0.355 & 0.496 & 0.546 & 0.741 & 0.760 & 0.522 & 0.460 & 0.489 & 0.568 & 0.527 & 0.568 \\ \hline
BERT &  & 0.834 & 0.492 & 0.649 & 0.693 & 0.850  & 0.863  &  0.895 & 0.875 & 0.890 & 0.890 & 0.908  & 0.909 \\
DuoBERT &  & 0.885 & 0.541 & 0.713 & 0.756 & 0.886 & 0.887 & 0.923 & \textbf{0.921} & 0.922 & 0.922 & 0.923 & 0.923  \\
BERT(Transfer) & \checkmark  & 0.714 & 0.361 & 0.504 & 0.553 & 0.742  & 0.764 &  0.642 & 0.567 & 0.612 & 0.623 & 0.668 & 0.719\\ \hline
Sentence-BERT & \checkmark & 0.750 & 0.404 & 0.543 & 0.589 & 0.810 & 0.861 &  0.794 & 0.701 & 0.775 & 0.785 & 0.864 & 0.905\\
RankSVM & \checkmark & 0.809 & 0.408 & 0.607 & 0.661 & 0.887 & 0.917 & 0.846 & 0.778 & 0.832 & 0.840  & 0.898 & 0.930\\
CTM  &  &  0.856 &	0.356 &	0.481 &	0.525 & 0.894 &	0.935 & 0.926 &	0.889 &	0.919 &	0.922 & 0.952 & 0.964 \\ \hline
\model & \checkmark & \textbf{0.902} & \textbf{0.542} & \textbf{0.741} & \textbf{0.798} & \textbf{0.934} &	\textbf{0.951} & \textbf{0.931}  & 0.899  & \textbf{0.926}  & \textbf{0.928}  & \textbf{0.957}  & \textbf{0.967} \\ \hline
\end{tabular}
}
\end{table*}

\begin{table*}[htbp]
\centering
\caption{Ablation study of \model. Best results are in \textbf{boldface}. AG: Ablation Group.}
\label{tab:ablation}
\setlength\tabcolsep{3.5pt}
{\small
\begin{tabular}{cl|cccccc|cccccc}
\hline
 \multirow{3}{*}{\textbf{AG}} & \multirow{3}{*}{\textbf{Variant}}  & \multicolumn{6}{c|}{\textbf{Weibo}} & \multicolumn{6}{c}{\textbf{Twitter}} \\ \cline{3-14}
 & &\multirow{2}{*}{MRR} & \multicolumn{3}{c}{MAP@} & \multicolumn{2}{c|}{HIT@} & \multirow{2}{*}{MRR} & \multicolumn{3}{c}{MAP@} & \multicolumn{2}{c}{HIT@} \\ \cline{4-8} \cline{10-14}
 & & & 1 & 3 & 5 & 3 & 5 &  & 1 & 3 & 5 & 3 & 5\\ \hline
- & \model & \textbf{0.902} & \textbf{0.542} & \textbf{0.741} & \textbf{0.798} & 0.934 &	0.951 & \textbf{0.931}  & 0.899  & \textbf{0.926}  & \textbf{0.928}  & \textbf{0.957}  & \textbf{0.967} \\ \hline
\ 1& \textit{w/o \texttt{ROUGE} guidance} & 0.892 &	0.535 &	0.729 &	0.786 &0.925 &	0.943  & 0.929  & \textbf{0.905}  & 0.924  & 0.926 & 0.945  & 0.952 \\ \hline
\ \multirow{3}{*}{2} & \textit{w/ rand mem init}  & 0.879 &	0.516 &	0.700 &	0.753 & 0.912 & 0.935 & 0.897 & 0.860 & 0.890 & 0.893 & 0.922  & 0.938 \\
\ & \textit{w/o mem update} &  0.898 & 0.541 & 0.736 & 0.790 & 0.935 &	0.948 & 0.925 &	0.897 &	0.860 &	0.890 & 0.922 &	0.938  \\
\ & \textit{w/o PMB} & 0.897  & 0.537  & 0.734  & 0.792 & 0.931  & 0.948 &  0.920 & 0.885 & 0.913 & 0.917 & 0.944 &0.960 \\\hline
\  \multirow{2}{*}{3} & \textit{w/ avg. pool} & 0.901 & 	0.540 	&0.739 & 0.796& \textbf{0.938} &	\textbf{0.958} & 0.923&0.892 &0.917 &0.919 & 0.944 &0.954\\
\ & \textit{w/o pattern aggr.} & 0.896 &	0.535 & 0.734 &	0.791 & 0.930 &	0.945 & 0.922 & 0.890 & 0.917 & 0.919  & 0.947 & 0.954 \\ \hline
\end{tabular}
}
\end{table*}

\subsection{Baseline Methods}\label{sec:baselines}

\noindent\textbf{BERT-based rankers from general IR tasks}

\textbf{BERT}~\citep{bert}: A method of pretraining language representations with a family of pretrained models, which has been used in general document reranking to predict the relevance.~\citep{reranking-with-bert, birch}

\textbf{DuoBERT}~\citep{duobert}: A popular BERT-based reranker for multi-stage document ranking. Its input is a query and a pair of documents. The pairwise scores are aggregated for final document ranking. Our first baseline, BERT (trained with query-article pairs), provides the inputs for DuoBERT.

\textbf{BERT(Transfer)}: As no sentence-level labels are provided in most document retrieval datasets, \citet{simple-bert} finetune BERT with short text matching data and then apply to score the relevance between query and each sentence in documents. The three highest scores are combined with BM25 score for document-level prediction.

\noindent\textbf{Rankers from related works of our task}

\textbf{Sentence-BERT}: \citet{shaar2020} use pretrained Sentence-BERT models to calculate cosine similarity between each sentence and the given claim. Then the top similarity scores are fed into a neural network to predict document relevance.

\textbf{RankSVM}: A pairwise RankSVM model for reranking using the scores from BM25 and sentence-BERT (mentioned above), which achieves the best results in \citep{shaar2020}.

\textbf{CTM}~\citep{vo2020}: This method leverages GloVe and ELMo to jointly represent the claims and the FC-articles for predicting the relevance scores. Its multi-modal version is not included as \model~focuses on key textual information.

\subsection{Experimental Setup}
\noindent\textbf{Evaluation Metrics.} As this is a binary retrieval task, we follow~\citet{shaar2020} and report Mean Reciprocal Rank (MRR), Mean Average Precision@$k$ (MAP@$k$, $k=1,3,5$) and HIT@$k$ ($k=3,5$). See equations in Appendix~\ref{apd:eval-metrics}.

\noindent\textbf{Implementation Details.} In \model, the ROT and ARP components have one and eleven Transformer layers, respectively. The initial parameters are obtained from pretrained BERT models\footnote{We use \texttt{bert-base-chinese} for Weibo and \texttt{bert-base-uncased} for Twitter.}. Other parameters are randomly initialized. The dimension of claim and sentence representation in ARP and pattern vectors are $768$. Number of Clusters in PMB $K$ is $20$. Following~\citep{shaar2020} and~\cite{vo2020}, we use $k_1=50$ candidates retrieved by BM25. $k_2=3$ (Weibo, hereafter, W) / $5$ (Twitter, hereafter, T) key sentences are selected. We use \texttt{Adam}~\citep{adam} for optimization with $\epsilon=10^{-6}, \beta_1=0.9, \beta_2=0.999$. The learning rates are $5\times 10^{-6}$ (W) and $1\times 10^{-4}$ (T). The batch size is $512$ for pretraining ROT, $64$ for the main task.  According to the quantiles on training sets, we set $t_{low}=0.252$ (W) / $0.190$ (T), $t_{high}=0.295$ (W) / $0.227$ (T). The following hyperparameters are selected according to the best validation performance: $\lambda_R=0.01$ (W) / $0.05$ (T), $\lambda_Q=0.6$, $\lambda_P=0.4$, and $\lambda_m=0.3$.
The maximum epoch is $5$. All experiments were conducted on NVIDIA V100 GPUs with PyTorch~\citep{pytorch}.
The implementation details of baselines are in Appendix \ref{apd:imp-baselines}.

\subsection{Performance Comparison}
To answer \textbf{EQ1}, we compared the performance of baselines and our method on the two datasets, as shown in Table \ref{tab:main_exp}. We see that: (1) \model~ourperforms all compared methods on the two datasets (the exception is only the MAP@1 on Twitter), which indicates that it can effectively find related FC-articles and provide evidence for determining if a claim is previously fact-checked. (2) For all methods, the performance on Weibo is worse than that on Twitter because the Weibo dataset contains more claim-sentence pairs (from multiple sources) than Twitter and is more challenging. Despite this, \model's improvement is significant. (3) BERT(Transfer), Sentence-BERT and RankSVM use transferred sentence-level knowledge from other pretext tasks but did not outperform the document-level BERT. This is because FC-articles have their own characteristics, which may not be covered by transferred knowledge. In contrast, our observed characteristics help \model~achieve good performance. Moreover, \model~is also efficiency compared to BERT(Transfer), which also uses 12-layer BERT and selects sentences, because our model uses only one layer for all sentences (other 11 layers are for key sentences), while all sentences are fed into the 12 layers in BERT(Transfer).

\subsection{Ablation Study}
To answer \textbf{EQ2}, we evaluated three ablation groups of \model's variants (AG1$\sim$AG3) to investigate the effectiveness of the model design.\footnote{We do not run MTM without sentence selection due to its high computational overhead which makes it unfeasible for training and inference.} Table~\ref{tab:ablation} shows the performance of variants and \model.

\textbf{AG1: With vs. Without \texttt{ROUGE}.}
 The variant removes the guidance of \texttt{ROUGE} (\model~\textit{w/o \texttt{ROUGE} guidance}) to check the effectiveness of \texttt{ROUGE}-guided finetuning. The variant performs worse on Weibo, but MAP@1 slightly increases on Twitter. This is probably because there are more lexical overlapping between claims and FC-articles in the Weibo dataset, while most of the FC-articles in the Twitter dataset choose to summarize the claims to fact-check.

\textbf{AG2: Cluster-based Initialization vs. Random Initialization vs. Without update vs. Without PMB.}
The first variant (\model~\textit{ w/ rand mem init}) uses random initialization and the second (\model~\textit{ w/o mem update}) uses pattern vectors without updating. The last one (\model~\textit{ w/o PMB}) removes the PMB. We see that the variants all perform worse than \model~on MRR, of which \textit{w/ rand mem init} performs the worst. This indicates that cluster-based initialization provides a good start and facilitates the following updates while the random one may harm further learning.

\textbf{AG3: Score-weighted Pooling vs. Average pooling, and With vs. Without pattern vector.} The first variant, \model~\textit{w/ avg. pool}, replace the score-weighted pooling with average pooling. The comparison in terms of MRR and MAP shows the effectiveness of using relevance scores as weights.
The second, \model~\textit{w/o pattern aggr.}, does not append the pattern vector to claim and sentence vectors before aggregation. It yields worse results, indicating the patterns should be taken into consideration for final prediction.

\subsection{Visualization of Memorized Patterns}
To probe what the PMB summarizes and memorizes, we selected and analyzed the key sentences corresponding to the residual embeddings around pattern vectors. Figure~\ref{fig:visualization} shows example sentences where highly frequent words are in boldface. These examples indicate that the pattern vectors do cluster key sentences with common patterns like ``...spread in WeChat Moments''.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.92\linewidth]{figs/visualization.pdf}
	\caption{Visualization of pattern vectors ($\blacktriangle$) and near residual embeddings (\ding{54}). The sentences are translated from Chinese.}
	\label{fig:visualization}
\end{figure}

\begin{figure}[tbp]
\centering
\includegraphics[width=0.92\linewidth]{figs/he1.pdf}
\caption{Results of human evaluation. (a)The proportion of the FC-articles where \model~found $\{0,1,2,3\}$ key sentences. (b) The proportion of key sentences at rank $\{1,2,3\}$. (c) The positional distribution of key sentences in the FC-articles.}
\label{fig:he}
\end{figure}

\subsection{Human Evaluation and Case Study}
The quality of selected sentences cannot be automatically evaluated due to the lack of sentence-level labels. To answer \textbf{EQ3}, we conducted a human evaluation. We randomly sampled 370 claim-article pairs whose articles were with over 20 sentences from the Weibo dataset. Then we showed each claim and top three sentences selected from the corresponding FC-article by \model. Three annotators were asked to check if an auto-selected sentence helped match the given query and the source article (i.e., key sentences). Figure~\ref{fig:he} shows (a) \model~hit at least one key sentence in 83.0\% of the articles; (b) 73.0\% of the sentences at Rank 1 are key sentences, followed by 65.1\% at Rank 2 and 56.8\% at Rank 3. This proves that \model~can find the key sentences in long FC-articles and provide helpful explanations. We also show the positional distribution in Figure~\ref{fig:he}(c), where key sentences are scattered throughout the articles. Using \model~to find key sentences can save fact-checkers' time to scan these long articles for determining whether the given claim was fact-checked.

Additionally, we exhibit two cases in the evaluation set in Figure~\ref{fig:cases}. These cases prove that \model~found the key sentences that correspond to the characteristics described in Section~\ref{sec:intro}. Please refer to~Appendix \ref{apd:error} for further case analysis.

\begin{figure}[t]
	\centering
	\includegraphics[width=\linewidth]{figs/cases}
	\caption{Cases in the set of human evaluation. Quotations are underlined and patterns are in {\color{red} \textbf{boldface}}.}
	\label{fig:cases}
\end{figure}

\section{Conclusions}

We propose \model~to select from fact-checking articles key sentences that introduce or debunk claims. These auto-selected sentences are exploited in an end-to-end network for estimating the relevance of the fact-checking articles w.r.t. a given claim. Experiments on the Twitter dataset and the Weibo dataset (newly constructed) show that \model ~outperforms the state of the art. Moreover, human evaluation and case studies demonstrate that the selected sentences provide helpful explanations of the results.

\section*{Acknowledgments}
The authors thank Guang Yang, Tianyun Yang, Peng Qi and anonymous reviewers for their insightful comments. Also, we thank Rundong Li, Qiong Nan, and other annotators for their efforts. This work was supported by the National Key Research and Development Program of China (2017YFC0820604), the National Natural Science Foundation of China (U1703261), and the Fundamental Research Funds for the Central Universities and the Research Funds of Renmin University of China (No. 18XNLG19). The corresponding authors are Juan Cao and Xirong Li.

\section*{Broader Impact Statement}
Our work involves two scenarios that need the ability to detect previously fact-checked claims: (1) For social media platforms, our method can check whether a newly published post contains false claims that have been debunked. The platform may help the users to be aware of the text's veracity by providing the key sentences selected from fact-checking articles and their links. (2) For manual or automatic fact-checking systems, it can be a filter to avoid redundant fact-checking work. When functioning well, it can assist platforms, users, and fact-checkers to maintain more credible cyberspace. But in the failure cases, some well-disguised claims may escape. This method functions with reliance on the used fact-checking article databases. Thus, authority and credibility need to be carefully considered in practice. We did our best to make the new Weibo dataset for academic purpose reliable. Appendix~\ref{apd:dataset} introduces more details.

\bibliographystyle{acl_natbib}


\section{Introduction}

Deep neural networks have achieved superior performance on numerous vision tasks \cite{Zisserman1,Zisserman2,semanticsegmentation}. However, they contain complicated black boxes with huge, unexplainable parameters that begin with random initialization and reach another unpredictable (still) sub-optimal point. Such transformations are non-linear for each problem, the interpretability remains unsolved.

The vision community relies on estimating saliency maps for deciphering the decision-making process of deep networks. We try to bridge the unknown gap between the input space and decision space through this saliency map. These saliency map generation approaches can be divided into different procedures regarding input space \cite{RISE,SISE_main,Ada-SISE}, feature maps \cite{CAM,Axiom-CAM,Grad-CAM,Grad-CAM++,FullGrad,integrated,Ablation-CAM}, or propagation scheme \cite{Guidedbackpropagation,DEEPLIFT,LRP,pointinggame,Thereandbackagain}. Usual perturbation-based methods
\cite{zeiler,externalperturbation1,externalperturbation2} probe the input space into different deviated versions and obtain a unified saliency map by their underlying algorithms. Even though their approach produces a better result, their process is relatively slow. CAM-like methods rely on the gradient for the given class and create a decent output within a very brief computation time.

\begin{figure*}[!htbp]
\centering
\begin{subfigure}[l]{0.80\textwidth}
    \centering
    \begin{subfigure}[t]{0.15\textwidth}
        \centering
        \includegraphics[height=15mm,width=\textwidth]{fig1/elephant_single_class.png}
       \subcaption{Single Class}
        \label{a}
    \end{subfigure}
    \begin{subfigure}[t]{0.15\textwidth}
        \centering
        \includegraphics[height=15mm,width=\textwidth]{fig1/single_class_feature_map.jpg}
       \subcaption{Feature map}
        \label{b}
    \end{subfigure}
    \begin{subfigure}[t]{0.15\textwidth}
        \centering
        \includegraphics[height=15mm,width=\textwidth]{fig1/single_class_ours.jpg}
       \subcaption{Ours}
        \label{c}
    \end{subfigure}
    \begin{subfigure}[t]{0.15\textwidth}
        \centering
        \includegraphics[height=15mm,width=\textwidth]{fig1/scorecam_single.jpg}
       \subcaption{\cite{Score-CAM}}
        \label{d}
    \end{subfigure}
    \begin{subfigure}[t]{0.15\textwidth}
        \centering
        \includegraphics[height=15mm,width=\textwidth]{fig1/single_class_grad_CAM.jpg}
       \subcaption{\cite{Grad-CAM}}
        \label{e}
    \end{subfigure}
    \begin{subfigure}[t]{0.15\textwidth}
        \centering
        \includegraphics[height=15mm,width=\textwidth]{fig1/single_class_gradcampp.jpg}
       \subcaption{\cite{Grad-CAM++}}
        \label{f}
    \end{subfigure}
    \subcaption{Feature map characteristics for a single class}
    \label{sinef}
    \centering
\end{subfigure}

\begin{subfigure}[r]{0.90\textwidth}
    \centering
          \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/input_image_duo_class.jpg}
       \subcaption{Dual Class}
        \label{g}
    \end{subfigure}
          \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/first_im.jpg}
       \subcaption{Feature map}
        \label{h}
    \end{subfigure}
          \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/fifth_im.jpg}
       \subcaption{Ours(\nth{1})}
        \label{i}
    \end{subfigure}
          \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/fifth_im.jpg}
       \subcaption{Ours(\nth{2})}
        \label{j}
    \end{subfigure}
          \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/duel_scorecam.jpg}
       \subcaption{\cite{Score-CAM}}
        \label{k}
    \end{subfigure}
    \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/third_im.jpg}
       \subcaption{\cite{Grad-CAM}(\nth{1})}
        \label{l}
    \end{subfigure}
    \begin{subfigure}[t]{0.13\textwidth}
        \centering
        \includegraphics[height=13mm,width=\textwidth]{fig1/forth_im.jpg}
       \subcaption{\cite{Grad-CAM++}(\nth{1})}
        \label{m}
    \end{subfigure}
    \subcaption{Feature map characteristics for dual class}
    \label{duof}
\end{subfigure}
\caption{Proposed global guidance maps $g_M$ for cat, proposed saliency map ($S_c$) with and without global guidance, and the same maps for the dog. The proposed global guidance map provides strong localization information as well as exclusion of non-target class.}
\label{algorithm figure}
\end{figure*}

This study explores the established framework of CAM-based methods to address current issues within vision-based interpretability. Similar studies usually rely on gradient information to produce saliency maps. Some replace the gradient dependency with score estimation to build the saliency map out of the feature maps. Nonetheless, these studies formulate saliency maps that often suffer degradation during class discriminative examples. Moreover, their weighted accumulation does not always cover the expected region for the given single class instance, as the associated weights sometimes do not address the local correspondence. On the other hand, the gradient-less methods take significant time to produce a score index for the feature maps.

Typically, the gradient maps are averaged into single values to produce the saliency map within CAM-like studies. However, this projection is not always efficient for the saliency map. In figure \ref{algorithm figure}, we can see that weighted multiplication still contains traces of unwanted classes. To address the weighted multiplication issue, we first look at the problem setup. For a given image, we use a model that produces a fixed number of feature maps before going into the dense layers. This fixation limits our search space for the saliency map. The only variables are the gradient maps that change according to the class.

Gradient matrix to a single value conversion eventually reduces many significant gradients from influencing the overall accumulation. If we keep all the gradients and perform elementwise multiplication with feature maps, the obtained map contains better representations of the assigned class. The acquired map is the intermediate global guidance map for the usual local weighted multiplication. We use this guidance map to constrain the generated feature maps to be responsive only to the assigned class through element-wise matrix multiplication. After that, we perform the formal weighted accumulation for acquiring the saliency map, followed by carefully designed upscaling process. Thus, the produced saliency map is more responsive to the given class and its boundaries than previous approaches. The following lines summarize our overall contributions:

\begin{itemize}
\item A new saliency map generation scheme is formulated by introducing global guidance map that incorporates elementwise influence of the gradient tensor onto the saliency map.
\item Acquired boundaries for the saliency maps are crisper than contemporary studies and perform efficiently within single/multi-class/multi-instance-single class cases.
\item To validate our study, we perform seven different metric analyses on three different datasets, and the proposed study achieves state-of-the-art performance in most cases.
\end{itemize}

\section{Related Work}
\textbf{Backprop-based methods.}
Zisserman \textit{et al}.\cite{Zisserman1,Zisserman2} first introduced gradient calculation by focusing on computing the confidence score for  explanation generation, and other back propagation-based explanation studies have been introduced \cite{Guidedbackpropagation,integrated,FullGrad}. However, their gradient employment and manipulation lead to several issues, as addressed by \cite{integrated,FullGrad}. Instead of a single layer gradient, Srinivas \textit{et al.} \cite{FullGrad} focuses on aggregating gradients from all convolutional layers.

\textbf{Activation-based methods.}
Class activation mapping (CAM) methods are based on the study \cite{CAM}, where authors select feature maps as a medium for creating an explanation map. GradCAM \cite{Grad-CAM} is a weighted linear combination of the feature maps followed by the ReLU operation for a given image and the respective model. Later, GradCAM++ \cite{Grad-CAM++} introduced the effect of the higher-order gradients with inclusive only to positive elements properties. In this way, they achieve more precise representation compared to the previous studies. However, gradients are not the only way to generate the saliency map, which inspires the ScoreCAM \cite{Score-CAM}, AblationCAM \cite{Ablation-CAM}. X-GradCAM \cite{Axiom-CAM}, an extension to the GradCAM, follows the same the underlying weighted multiplication for GradCAM. EigenCAM \cite{Eigen-CAM} introduces principal component analysis into developing the saliency map. CAMERAS \cite{Cameras} utilizes basic CAM studies with multiscale input extension, leveraging the fusion technique of multiscale feature and gradient map weighted multiplication.

\textbf{Perturbation-based methods.}
Another group of studies treat the neural network as a ``white box" instead of a ``black box" and propose an explanation map by probing the input space. These studies generate saliency maps by checking the response upon the manipulated input space. By blocking/blurring/masking some of the regions randomly, these studies observe the forward pass response for each case and finally aggregate their decision to develop the saliency map. RISE \cite{RISE} first introduces such analysis, followed by external perturbation \cite{externalperturbation1,externalperturbation2}  where authors rely upon optimization procedure. SISE \cite{SISE_main} presents feature map selection from multiple layers, followed by attribution map generation and mask scoring to generate the saliency map. Later, they improvise it through adaptive mask selection in the ADA-SISE \cite{Ada-SISE} study.

\section{Proposed methodology}
This section will describe the formulation of the proposed approach as shown in figure \ref{architecture} for obtaining the saliency for the given model and image.

\subsection{Baseline formulation }
Every trained model infers through the collective response of its feature maps for the given image. The current understanding of deep learning requires more research to define the ideal formulation for the extracted feature maps during inference. During inference, activated feature maps can be sparse, shallow, and rich in containing the spatial correspondence and their collective distribution governs the outcome of the final prediction.

Let $\mathcal{\phi}$ be the model we take for the inference. For any image $\mathcal{X} \in \mathcal{R}^{W \times H \times 3}$, model $\mathcal{\phi}$ generates feature maps $\mathcal{M}$. Say, $\mathcal{M}^{k_l}$ is the $k^{th}$ feature channel at $l^{th}$ layer, before the final dense layer. If we aggregate them all, we will obtain a unified map that will represent the collective correspondence due to the model $\mathcal{\phi}$ for the given image. This aggregation for the activated channels is as follows:

\begin{equation}\label{eq1}
   \mathcal{A}^{l} = \sum_{k} \mathcal{M}^{k_l}
\end{equation}

\begin{figure*}[!htbp]\label{f4}
	\centering
	\includegraphics[width = 6 in, height= 2.5 in]{fig1/architecture_final.png}
	\caption{Visual comparison between previous state-of-the-art studies and the proposed method. For the given demonstrations, our approach can mark down the primary salient regions under challenging visual conditions. Additionally, our saliency maps are more concrete and leaves almost no traces for the secondary-salient areas.}
	\label{architecture}
\end{figure*}

This tensor $\mathcal{A}^{l}$ contains the global representation for all activated feature maps, which can serve the purpose of marking salient regions with careful tuning as shown in figures \ref{sinef}, and \ref{duof}. In equation (\ref{eq1}),  we aggregated all of the feature representations into $\mathcal{A}^{l}$; elements over a specific threshold may correspond to the primary class information. Nonetheless, this observation only works within images with a single class, but is not appropriate for the dual class scenarios, as shown in figure \ref{algorithm figure}.

To achieve class discriminatory behaviour, researchers \cite{Grad-CAM,Grad-CAM++,integrated,FullGrad} worked with the idea of using weighted aggregation for equation (\ref{eq1}), in contrast to plain linear addition operation. To extract weights, first they compute the gradient maps  $\nabla_\mathcal{C}$ with respect to the given class $\mathcal{C}$, from the feature maps $\mathcal{M}$.

If $Y^{\mathcal{C}}$ is the class score \cite{Grad-CAM} for the given image from the input model $\mathcal{\phi}$, for each location $(i,j)$ of the $k^{th}$ feature map at the $l^{th}$ layer, then the corresponding gradient map is expressed as :

\begin{equation}\label{eq2}
   \nabla_{\mathcal{C}_{i j}}^{k_{l}} =  \frac{\partial Y^{\mathcal{C}}}{\partial \mathcal{M}_{i j}^{k_{l}}}
\end{equation}

If each feature map holds $\mathcal{Z}$ number of elements, then the corresponding weight for each feature map $\mathcal{M}^{k_l}$ is:

\begin{equation}\label{eq3}
     \lambda_{\mathcal{C}}^{k_{l}}= \frac{1}{\mathcal{Z}}\sum_{i} \sum_{j} \nabla_{\mathcal{C}_{i j}}^{k_{l}}
\end{equation}

which is the mean value of $\nabla_{\mathcal{C}}^{k_{l}}$. Hence, the regular baseline formulation \cite{Grad-CAM} for the saliency map  $\mathcal{S}_{\mathcal{C}}$ estimation is expressed as follows:

\begin{equation}\label{eq4}
    \mathcal{S}_{\mathcal{C}} =  \textit{ReLU}({\sum_{k}    \lambda_{\mathcal{C}}^{k_{l}}\times \mathcal{M}^{k_l}})
\end{equation}

\subsection{Incorporating global guidance }
Equation (\ref{eq4}) achieves class discriminatory behavior, however it still faces challenges due to its formulation. If we investigate the above equation, we see that $\lambda_{\mathcal{C}}^{k_{l}}$  weighs the corresponding feature map $\mathcal{M}^{k_l}$. Typical $\lambda_{\mathcal{C}}^{k_{l}}$ treats every member of the given  $\mathcal{M}^{k_l}$ equally and increases/decreases their collective effect homogeneously. Therefore, we can still see the traces of the other classes in the saliency map, as shown in figure \ref{algorithm figure}. Other studies \cite{SISE_main,Ada-SISE} have shown class discriminatory performance through perturbations without relying upon the gradients. However, those studies are time expensive \cite{FullGrad} and often require human interaction. Hence, it is natural to ask, can we still rely on the gradient-weighted operation and address the above issues?

In response to this, we propose global guidance map. To obtain the global guidance map, we perform a simple elementwise multiplication between the feature maps and their corresponding gradient maps. The formulation for the global guidance map is as follows:

\begin{equation}\label{eq5}
    \mathcal{G}_{M} =  \textit{ReLU}({\sum_{k}    \nabla_{\mathcal{C}}^{k_{l}}\odot \mathcal{M}^{k_l}})
\end{equation}

\begin{figure*}[!htbp]
\centering
\begin{subfigure}[l]{0.18\textwidth}
    \centering
    \includegraphics[height=24mm,width=30mm]{rebuttal/rinput_image_duo_class.jpg}
    \subcaption{Input}
    \label{fig:a}
    \centering
\end{subfigure}
\begin{subfigure}[r]{0.70\textwidth}
    \centering
    \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/ggct.jpg}
       \subcaption{$g_M$(cat)}
        \label{fig:b}
    \end{subfigure}
        \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/prp.jpg}\\
        \subcaption{$S_c$ w/o $g_M$(dog)}
        \label{fig:c}
    \end{subfigure}
     \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/catpp.jpg}\\
        \caption{$S_C$ w/o $g_M$ for \cite{Grad-CAM++}(cat)}
        \label{fig:d}
    \end{subfigure}\\
    \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/ggdg.jpg}\\
        \caption{$g_M$ (dog)}
        \label{fig:e}
    \end{subfigure}
        \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/prpdg.jpg}\\
        \caption{$S_C$ w/o $g_M$(dog)}
        \label{fig:f}
    \end{subfigure}
        \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/woggprp.jpg}\\
        \caption{$S_C$ w/o $g_M$(dog)}
        \label{fig:g}
    \end{subfigure}
        \begin{subfigure}[t]{0.25\textwidth}
        \centering
        \includegraphics[height=22mm,width=\textwidth]{rebuttal/dogppp.jpg}\\
        \caption{$S_C$ w/o $g_M$ for \cite{Grad-CAM++}(dog)}
        \label{fig:h}
    \end{subfigure}

\end{subfigure}
\caption{Proposed global guidance maps $g_M$ for cat, proposed saliency map ($S_c$) with and without global guidance
(first row), and the same maps for the dog [second row and third row]. The proposed global guidance map provides strong localization
information as well as exclusion of non-target class.}
\label{rebuttal}
\end{figure*}

The idea behind considering the global guidance map is to focus only on the salient regions by limiting the operating zone for the $\lambda_{\mathcal{C}}^{k_{l}}$. Since the multiplication operation  of equation (\ref{eq5}) investigates individual responses for every element of the gradient maps, we can mark the class-specific regions from the feature maps. In this way, captured guidance map successfully omits the trace of the secondary classes from equation (\ref{eq4}) unless the generated feature maps heavily overlap between categories, signifying the possible misclassification.

In summary, we first compute $\mathcal{G}_{M}$ and multiply it with each of the feature maps to obtain the class discriminative feature maps from the initial class  representative feature maps. Then, we perform the weighted multiplication between each member of the $\lambda_{\mathcal{C}}^{k_{l}}$ and class discriminative feature maps, followed by the final aggregation to obtain the desired representation. Hence, our proposed weighted-multiplicative-aggregation is as follows:

\begin{equation}\label{eq6}
   \mathcal{S}_{\mathcal{C}} =  \textit{ReLU}({\sum_{k}    \lambda_{\mathcal{C}}^{k_{l}}\times (\mathcal{G}_{M}\odot\mathcal{M}^{k_l}}))
\end{equation}

In equation (\ref{eq6}), the only reason we are using the $\lambda_{\mathcal{C}}^{k_{l}}$ is to gain a similar homogeneous increment as in equation (\ref{eq4}), which also helps to preserve visual integrity during the final upsampling operation. Since guidance map $\mathcal{G}_{M}$ can successfully omit the secondary classes from any $k^{th}$ feature map $\mathcal{M}^{k_l}$ by \lq masking\rq\space the primary governing region, it also becomes possible to produce increments in the desired class region with the additional multiplication help from $\lambda_{\mathcal{C}}^{k_{l}}$. Finally, typical smoothing and normalization are performed on the saliency map before post aggregation up-sampling onto the given image size. In this way, the achieved saliency map $\mathcal{S}_{\mathcal{C}}$ shows significant improvement in both single class representative and multiclass discriminative cases.

\begin{figure*}[!htbp]
	\centering
	\includegraphics[width = 5.8 in, height= 3.3 in]{fc/comparison.png}
	\caption{Visual comparison between previous state-of-the-art studies and the proposed method. For the given demonstrations, our approach can mark down the primary salient regions under challenging visual conditions. Additionally, our saliency maps are more concrete and leaves almost no traces for the secondary-salient areas.}
	\label{qualitative result}
\end{figure*}

\section{Performance evaluation}
\textbf{Datasets.}
Our experimental setup covers three widely used vision datasets: ImageNet, MS-COCO 14, and PASCAL-VOC 12. Among them, PASCAL-VOC 12 provides the full segmentation annotations for the input images. Hence, experiments using segmentation-oriented metrics are from this dataset. Other experiments do not involve segmentation labels, as the rest of the experiments are applicable for all datasets mentioned above. For the ImageNet and MS-COCO 14 datasets, we randomly selected a few thousand \cite{Score-CAM,Grad-CAM++,Axiom-CAM} for the experiments.

\textbf{Compared studies.}
For visual comparison, given the availability and functional complexities, we consider available official implementations of the GradCAM\cite{Grad-CAM}, GradCAM++\cite{Grad-CAM++}, X-gradCAM\cite{Axiom-CAM}, ScoreCAM\cite{Score-CAM}, Fullgrad\cite{FullGrad}, Smooth-Fullgrad\cite{FullGrad}, CAMERAS\cite{Cameras}, Integrated\cite{integrated}, and Relevance-CAM\cite{relevance}. We report seven different quantitative analyses in addition to the visual demonstration. We  extract saliency maps from pretrained VGG16 \cite{VGG16} and ResNet50\cite{ResNet50} networks for all of the methods above in our experiments. A later section reports the quantitative results from ResNet50 in the respective tables. As with previous studies \cite{Score-CAM,integrated,Grad-CAM++,Axiom-CAM}, few thousand random images are selected from the datasets for our experiments. However, since the random images from previous studies are not made public, the results of our random selections may vary from the cited studies.

\begin{figure*}[htbp]
\begin{center}
  \begin{subfigure}{0.7\textwidth}
    \includegraphics[width=5in,height=1.8in]{fig4/alggggg.PNG}
    \caption{ Interpretation comparison between proposed and GradCAM++\cite{Grad-CAM++}}
    \label{4.1}
  \end{subfigure}
\end{center}
\begin{center}
  \begin{subfigure}{0.7\textwidth}
    \includegraphics[width=5in,height=0.6in]{fig4/algo_crop.pdf}\\
   \caption{``Ruddy Turnstone" images from the ``ImageNet" dataset.}
    \label{4.2}
  \end{subfigure}
\end{center}
  \caption{Here, (a) shows the interpretation comparison between the proposed and GradCAM++\cite{Grad-CAM++}. The upper row is the response map for the primary ``Albatross" bird class, and the second row is for the secondary class, ``Ruddy Turnstone" bird. The proposed method clearly presents the difference between VGG16 and ResNet50 in terms of interpretation, whereas GradCAM++ responses are all similar across different networks and classes.
  (b) As we suspect that dataset bias might lead to such a decision for the given ``Albatross" images, an inspection of the ImageNet dataset could clarify further. Upon examining typical Ruddy Turnstone bird images in the ImageNet dataset, we see that stony shore is the background for most of the Ruddy Turnstone bird images. }
  \label{4}
\end{figure*}

\subsection{Visual demonstration}
In this section, we show a qualitative comparison with previous methods. In figure \ref{qualitative result}, we have included diverse sets of images from the datasets. The first five rows show the performance due to ResNet50, and later are for VGG16. Our image selection includes class representative, class discriminative, and multiple instances examples. Here, the proposed method can capture the class discriminative region with greater confidence, if not the entire class area itself for the bicycle, sheep, sea-bird, and dog image. Our scheme captures the class region for the sea-bird image and excludes its reflection in the water compared to other methods.

Additionally, the proposed CAM bounds the whole dog class as a dog and captures the sheep in a lowlight environment more clearly than other methods. For images with dual classes, our method presents superior class discriminative performance. Our study successfully bounds the horse regions in the horse images without leaving any traces to other class regions. On the other hand, compared studies often mark both horse and secondary class instances.

\subsection{Quantitative analysis}
To present our quantitative analysis, we perform the following experiments: model's performance drops and increments due the to salient and context regions, Pointing score, Dice score, and IoU score. We present the scores for the above metrics for ResNet50 for all datasets.

\textbf{Performance due to the saliency region.}
If we have a perfect model and a perfect interpreter to mark the spatial correspondence for the specific class, the network will provide a similar prediction for both the given image and the segmented salient image. Here, we first extract the salient region from the given images with the help of the given interpreter. Then we perform prediction on the original image, and the corresponding salient image \cite{Grad-CAM++} and check the performance drop for the given interpreter. The expectation is that the better interpreter can exclude the non-salient region as much as possible; hence, the performance drop will be as low as possible. Therefore, our first metric delivers the performance drop due to salient area only as input. For some cases, prediction performance hinders due to the presence of strong spatial context.

\begin{table*}[!htbp]
\centering
\caption{Comparative evaluation in terms of salience zone drop and context zone increase, Pointing Game, Dice, IoU (higher is better) and salience zone increase and context zone drop (lower is better) on the PASCAL VOC 2012 dataset for ResNet50 model.
The best scores are in bold form and second best scores are  underlined.}
\resizebox{\textwidth}{!}{
\begin{tabular}{lcccccccccc}
\hline
                                 & GradCAM\cite{Grad-CAM} & GradCAM++\cite{Grad-CAM++} & X GradCAM\cite{Axiom-CAM} & CAMERAS\cite{Cameras} & FullGrad\cite{FullGrad} & SmoothFullGrad\cite{FullGrad} & Integrated\cite{integrated} & ScoreCAM\cite{Score-CAM}
                            & Relevance\cite{relevance}
                            & Proposed \\
\hline

Increase for salience zone $\uparrow $     & 0.0366 & \underline{0.0716} & 0.0366 & 0.0715 & 0.0421 & 0.0421 & 0.0172 & 0.0506 & 0.0559&\textbf{0.0786} \\
Drop for context zone $\uparrow$           & 0.9395 & 0.8812 & \underline{0.9429} & 0.8834 & 0.9083 & 0.9281 & 0.8124 & 0.9389 & 0.9089&\textbf{0.9443} \\
Pointing Game $\uparrow$                   & 0.3355 & 0.4731 & 0.3733 & 0.4412 & 0.3713 & 0.4422 & 0.2642 & 0.4322& \underline{0.532}&\textbf{0.5945}  \\
Dice  $\uparrow$                           & 0.2822 & 0.3422 & 0.2934 & 0.3342 & 0.2942 & 0.3328 & 0.1834 & 0.3321 & \underline{0.411}&\textbf{0.4321} \\
IoU   $\uparrow$                         & 0.0823 & 0.1122 & 0.0901 & 0.1007 & 0.0812 & 0.0963 & 0.0625 & 0.0943 & \underline{0.121}&\textbf{0.1321} \\
Drop for salience zone $\downarrow$        & 0.8996 & \underline{0.8064} & 0.8745 & 0.8201 & 0.8873 & 0.8762 & 0.9399 & 0.8567 & 0.8333&\textbf{0.7784} \\
Increase for context zone $\downarrow $    & 0.0172 & 0.0312 & 0.0205 & 0.0244 & 0.0291 & 0.0215 & 0.0411 & \textbf{0.0151} &0.029& \underline{0.0183} \\

\hline
\end{tabular}
}
\centering

\label{t1}
\end{table*}

\begin{table*}[!htbp]

\caption{ Comparative performance drop and increment of saliency and context zones on the MS-COCO 14 dataset for ResNet50 model.}

\centering

\resizebox{\textwidth}{!}{
\begin{tabular}{lcccccccccc}
\hline

                                          & GradCAM\cite{Grad-CAM} & GradCAM++\cite{Grad-CAM++} & X GradCAM\cite{Axiom-CAM} & CAMERAS\cite{Cameras} & FullGrad\cite{FullGrad} & SmoothFullGrad\cite{FullGrad} & Integrated\cite{integrated} & ScoreCAM\cite{Score-CAM} &
                                          Relevance\cite{relevance}&                           Proposed \\
\hline
Drop for context zone $\uparrow$            & 0.9023 & \textbf{0.9495} & 0.9142 & \underline{0.9424} & 0.8961 & 0.9025 & 0.8607 & 0.9183 & 0.9081 & 0.9391 \\
Increase for saliency zone $\uparrow$       & 0.0490 & \underline{0.0913} & 0.0555 & 0.0935 & 0.0455 & 0.0495 & 0.0796 & 0.0695 & 0.089  & \textbf{0.0999} \\
Drop for saliency zone $\downarrow$         & 0.8394 & \underline{0.7243} & 0.8081 & 0.7325 & 0.8454 & 0.8288 & 0.7713 & 0.7822 & 0.7357 & \textbf{0.6493} \\
Increase for context zone $\downarrow$      & 0.0245 & \underline{0.0165} & 0.0215 & 0.0172 & 0.0311 & 0.0315 & 0.0335 & 0.0205 & 0.0311 & \textbf{0.0152} \\
\hline
\end{tabular}
}
\centering

\label{t2}

\end{table*}

\begin{table*}[!htbp]
\caption{ Comparative performance drop and increment of saliency and context zones on the ImageNet
dataset for ResNet50 model.}
\centering

\resizebox{\textwidth}{!}{
\begin{tabular}{c c c c c c c c c c c}
\hline

                                          & GradCAM\cite{Grad-CAM} & GradCAM++\cite{Grad-CAM++} & X GradCAM\cite{Axiom-CAM} & CAMERAS\cite{Cameras} & FullGrad\cite{FullGrad} & SmoothFullGrad\cite{FullGrad} & Integrated\cite{integrated} & ScoreCAM\cite{Score-CAM} &
                                    Relevance\cite{relevance}&      Proposed \\
\hline
Drop for context zone $\uparrow$            & 0.8767 & 0.9178 & 0.8698 &\underline{0.9335} & 0.8379 & 0.8548 & 0.8938 & 0.8585 & 0.8541 &\textbf{0.9392} \\
Increase for saliency zone $\uparrow$       & 0.0535 & 0.0903 & 0.0635 & 0.0936 & 0.0803 & 0.0669 & 0.0435 & \underline{0.1003} & 0.0969 &\textbf{0.1008} \\
Drop for saliency zone $\downarrow$        & 0.7906 & \underline{0.6717} & 0.7682 & 0.7302 & 0.7775 & 0.7844 & 0.7267 & 0.6975 & 0.6844 &\textbf{0.6492} \\
Increase for context zone $\downarrow$      & 0.0234 & \underline{0.0067} & 0.0368 & 0.0134 & 0.0502 & 0.0301 & 0.0301 & 0.0468 & 0.0602 &\textbf{0.0012} \\
\hline
\end{tabular}
}
\centering

\label{t3}

\end{table*}

\textbf{Performance due to the context region.}
As above, if saliency extraction is as good as one expects, then we can set up another experiment with the context region. In this setup, we first exclude the salient part from the given image to obtain the context image and predict it. If the interpreter can successfully extract all the salient areas, the performance will drop near 100 percent.

\textbf{Pointing game, Dice Score, IoU.}
For image sets with segmenation labels, various segmentation evaluations can be calculated for saliency maps. We follow \cite{pointinggame} to perform the pointing game for class discriminative evaluation. In this performance metric, the ground-truth label is used to trigger each visualization approach, and the maximum active spot on the resulting heatmap is extracted. After that, it determines if the highest saliency points undergo into the annotated boundary box of an object, determining whether it is a hit or miss. The term \begin{tiny}$\frac{ Hits_{total}}{Hits_{total}+ Misses_{total}}$\end{tiny} is calculated as the pointing game accuracy, which high value represents a better explanation for any model. Dice score is a popular metric for analyzing segmentation performance. It results from the ratio of the doubled intersection over the total number of elements associated with this instance. IoU stands for intersection over the union. It is a widespread metric for evaluating segmentation performance. This score ranges from 0 to 1 and signifies the overlapping area between the obtained image and its corresponding ground truth.

\begin{figure*} [htbp]
\centering
\begin{tabular}{cccccc}

\includegraphics[width=0.8in,height =0.8 in]{fig5/ship_input.jpg}&
\includegraphics[width=1.8in,height =1.2 in]{fig5/ship_iauc.png} &
\includegraphics[width=1.8in,height =1.2 in]{fig5/ship_dauc.png} \\
\tiny 5.1(a)  & \tiny 5.1(b) & \tiny 5.1(c)  \\[1pt]
\end{tabular}
\\
\begin{tabular}{cccc}
\includegraphics[width=0.8in,height =0.8in]{fig5/dog_input.jpg} &
\includegraphics[width=1.8 in,height =1.2 in]{fig5/dog_iauc.png} &
\includegraphics[width=1.8 in,height =1.2 in]{fig5/dog_dauc.png} \\
\tiny 5.2(d)  & \tiny 5.2(e)  & \tiny 5.2(f)\\[1pt]
\end{tabular}
\\
\begin{tabular}{cccc}
\includegraphics[width=0.8in,height =0.8in]{fig5/cycle_input.jpg} &
\includegraphics[width=1.8 in,height =1.2 in]{fig5/cycle_dauc.png} &
\includegraphics[width=1.8 in,height =1.2 in]{fig5/cycle_iauc.png} \\
\tiny 5.3(g)  & \tiny 5.3(h)  & \tiny 5.3(i)\\[1pt]

\end{tabular}

\caption{ AUC demonstration for the insertion and deletion operation for images on the left side. The above analysis shows that the proposed method can capture the most salient regions for a single class, dual-class, and dual-class with multiple instances compared to the previous methods \cite{Grad-CAM,Axiom-CAM,Grad-CAM++}.}

\label{fig:AUC2}
\end{figure*}

The proposed method performs better or similar to the best performing saliency generation method in table \ref{t1}.  Here, we present the comparative data on the Pascal VOC 2012 dataset for the ResNet50 model. Out of seven different performance tests, our method obtains the highest score for the six of them. For an increase in the context zone, our study differs only 0.03 from the best performing result. In table \ref{t2}, the proposed research avails state-of-the-art performance for three out of four metrics. Table \ref{t3} also shows the best performance of our method for every experiment metrics using the ImageNet dataset. The PASCAL VOC 2012 dataset has corresponding segmentation mask, and we can achieve pseudo segmentation masks by thresholding the saliency maps. Achieved scores for the Pointing game, Dice and IoU signify that our study can better capture interest zones than the compared studies. To measure the explainability, we also have conducted a comparative analysis in figure \ref{fig:AUC2} on three images from the Pascal VOC 2012 dataset and presented the insertion and deletion operation.  Here, our method captures the most salient regions for both single, dual, and dual classes with multiple instances in comparison to GradCAM \cite{Grad-CAM}, Grad-CAM++\cite{Grad-CAM++}, and X-GradCAM\cite{Axiom-CAM}.

\subsection{Interpretation comparison}
Saliency map generation is not all about capturing the class of interest as precisely as possible. A faithful interpretation is also a significant part of saliency generation studies. In other words, any interpretable method should explain why the underlying model is making such a prediction by marking the corresponding image region. However, we cannot present this for every image from the dataset, but a sophisticated example can show the difference from previous methods.

In figure \ref{4.1}, we have presented the interpretation comparison between the proposed study and the GradCAM++\cite{Grad-CAM++}. Here, the top-1 class response is \lq Albatross\rq\space for the given image. For VGG16, saliency map for the proposed method marks one of the Albatross birds and the water as context, but fails to mark other Albatross bird. For \lq Albatross\rq, our method with the VGG16 shows different interpretations due to the global guidance map and responds to the water as context. In contrast to ResNet50, our guidance map marks both Albatross birds without marking the water context. However, for both networks, GradCAM++\cite{Grad-CAM++} captures both of the birds also barely touches the water context. For this particular image, the proposed method presents clearer interpretation difference between VGG16 and ResNet50.

In \ref{4.2}, we show why the models might identify the given image as \lq Ruddy Turnstone\rq\space class. With our scheme, we interpret that surrounding stones and water are features that are corresponding to Ruddy Turnstone bird for both VGG16 and ResNet50. This interpretation makes more sense if we look at typical Ruddy Turnstone images from the ImageNet dataset \ref{4.2}, where the most of Ruddy Turstone birds are shown with stony sea-shore area as the background. Hence, we can utilize this interpretation as a medium for identifying the dataset bias. On the other hand, GradCAM++\cite{Grad-CAM++} shows the Albatross bird regions as the interpretation for the Ruddy Turnstone, and almost ignores the associated context.

\section{Conclusion}
In this study, we present a novel extension of the traditional gradient-dependent saliency map generation scheme. The proposed method leverages element-wise multiplicative aggregation as guidance with previous weighted multiplicative summation and further improves the performance of salient region bounding. Additionally, we showed our study's advanced class discriminative performance and presented evidence for better area framing with deeper networks. Furthermore, our model can avail crisper saliency map and significant quantitative improvement over three widely used datasets. We aim to integrate this study into other vision tasks in future work.

\bibliographystyle{unsrt}

\section{Introduction}
\lipsum[2]
\lipsum[3]

\section{Headings: first level}
\label{sec:headings}

\lipsum[4] See Section \ref{sec:headings}.

\subsection{Headings: second level}
\lipsum[5]
\begin{equation}
\xi _{ij}(t)=P(x_{t}=i,x_{t+1}=j|y,v,w;\theta)= {\frac {\alpha _{i}(t)a^{w_t}_{ij}\beta _{j}(t+1)b^{v_{t+1}}_{j}(y_{t+1})}{\sum _{i=1}^{N} \sum _{j=1}^{N} \alpha _{i}(t)a^{w_t}_{ij}\beta _{j}(t+1)b^{v_{t+1}}_{j}(y_{t+1})}}
\end{equation}

\subsubsection{Headings: third level}
\lipsum[6]

\paragraph{Paragraph}
\lipsum[7]

\section{Examples of citations, figures, tables, references}
\label{sec:others}
\lipsum[8] \cite{kour2014real,kour2014fast} and see \cite{hadash2018estimate}.

The documentation for \verb+natbib+ may be found at
\begin{center}
  \url{http://mirrors.ctan.org/macros/latex/contrib/natbib/natnotes.pdf}
\end{center}
Of note is the command \verb+\citet+, which produces citations
appropriate for use in inline text.  For example,
\begin{verbatim}
   \citet{hasselmo} investigated\dots
\end{verbatim}
produces
\begin{quote}
  Hasselmo, et al.\ (1995) investigated\dots
\end{quote}

\begin{center}
  \url{https://www.ctan.org/pkg/booktabs}
\end{center}

\subsection{Figures}
\lipsum[10]
See Figure \ref{fig:fig1}. Here is how you add footnotes. \footnote{Sample of the first footnote.}
\lipsum[11]

\begin{figure}
  \centering
  \fbox{\rule[-.5cm]{4cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
  \caption{Sample figure caption.}
  \label{fig:fig1}
\end{figure}

\subsection{Tables}
\lipsum[12]
See awesome Table~\ref{tab:table}.

\begin{table}
 \caption{Sample table title}
  \centering
  \begin{tabular}{lll}
    \toprule
    \multicolumn{2}{c}{Part}                   \\
    \cmidrule(r){1-2}
    Name     & Description     & Size ($\mu$m) \\
    \midrule
    Dendrite & Input terminal  & $\sim$100     \\
    Axon     & Output terminal & $\sim$10      \\
    Soma     & Cell body       & up to $10^6$  \\
    \bottomrule
  \end{tabular}
  \label{tab:table}
\end{table}

\subsection{Lists}
\begin{itemize}
\item Lorem ipsum dolor sit amet
\item consectetur adipiscing elit.
\item Aliquam dignissim blandit est, in dictum tortor gravida eget. In ac rutrum magna.
\end{itemize}

\section{Conclusion}
Your conclusion here

\section*{Acknowledgments}
This was was supported in part by......

\bibliographystyle{unsrt}