Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Beyond Nystr\"omformer -- Approximation of self-attention by Spectral
  Shifting   Transformer is a powerful tool for many natural language tasks which is based
on self-attention, a mechanism that encodes the dependence of other tokens on
each specific token, but the computation of self-attention is a bottleneck due
to its quadratic time complexity. There are various approaches to reduce the
time complexity and approximation of matrix is one such. In Nystr\"omformer,
the authors used Nystr\"om based method for approximation of softmax. The
Nystr\"om method generates a fast approximation to any large-scale symmetric
positive semidefinite (SPSD) matrix using only a few columns of the SPSD
matrix. However, since the Nystr\"om approximation is low-rank when the
spectrum of the SPSD matrix decays slowly, the Nystr\"om approximation is of
low accuracy. Here an alternative method is proposed for approximation which
has a much stronger error bound than the Nystr\"om method. The time complexity
of this same as Nystr\"omformer which is $O\left({n}\right)$.

24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg
16
The categories of this paper are: stat
8
The categories of this paper are: hep-ex
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
36
The categories of this paper are: cmp-lg
26
The categories of this paper are: bayes-an
19
The categories of this paper are: eess
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
33
The categories of this paper are: plasm-ph
27
The categories of this paper are: comp-gas