Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: An Empirical Study on Explanations in Out-of-Domain Settings   Recent work in Natural Language Processing has focused on developing
approaches that extract faithful explanations, either via identifying the most
important tokens in the input (i.e. post-hoc explanations) or by designing
inherently faithful models that first select the most important tokens and then
use them to predict the correct label (i.e. select-then-predict models).
Currently, these approaches are largely evaluated on in-domain settings. Yet,
little is known about how post-hoc explanations and inherently faithful models
perform in out-of-domain settings. In this paper, we conduct an extensive
empirical study that examines: (1) the out-of-domain faithfulness of post-hoc
explanations, generated by five feature attribution methods; and (2) the
out-of-domain performance of two inherently faithful models over six datasets.
Contrary to our expectations, results show that in many cases out-of-domain
post-hoc explanation faithfulness measured by sufficiency and comprehensiveness
is higher compared to in-domain. We find this misleading and suggest using a
random baseline as a yardstick for evaluating post-hoc explanation
faithfulness. Our findings also show that select-then predict models
demonstrate comparable predictive performance in out-of-domain settings to
full-text trained models.

34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
15
The categories of this paper are: nucl-ex
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio
28
The categories of this paper are: alg-geom
4
The categories of this paper are: cond-mat
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
9
The categories of this paper are: nlin
3
The categories of this paper are: physics