Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision
  Transformers   The self-attention-based model, transformer, is recently becoming the leading
backbone in the field of computer vision. In spite of the impressive success
made by transformers in a variety of vision tasks, it still suffers from heavy
computation and intensive memory costs. To address this limitation, this paper
presents an Interpretability-Aware REDundancy REDuction framework (IA-RED$^2$).
We start by observing a large amount of redundant computation, mainly spent on
uncorrelated input patches, and then introduce an interpretable module to
dynamically and gracefully drop these redundant patches. This novel framework
is then extended to a hierarchical structure, where uncorrelated tokens at
different stages are gradually removed, resulting in a considerable shrinkage
of computational cost. We include extensive experiments on both image and video
tasks, where our method could deliver up to 1.4x speed-up for state-of-the-art
models like DeiT and TimeSformer, by only sacrificing less than 0.7% accuracy.
More importantly, contrary to other acceleration approaches, our method is
inherently interpretable with substantial visual evidence, making vision
transformer closer to a more human-understandable architecture while being
lighter. We demonstrate that the interpretability that naturally emerged in our
framework can outperform the raw attention learned by the original visual
transformer, as well as those generated by off-the-shelf interpretation
methods, with both qualitative and quantitative results. Project Page:
http://people.csail.mit.edu/bpan/ia-red/.

17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
18
The categories of this paper are: econ
23
The categories of this paper are: solv-int
27
The categories of this paper are: comp-gas
15
The categories of this paper are: nucl-ex
8
The categories of this paper are: hep-ex
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
2
The categories of this paper are: math
10
The categories of this paper are: q-bio
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th