Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: From block-Toeplitz matrices to differential equations on graphs:
  towards a general theory for scalable masked Transformers   In this paper we provide, to the best of our knowledge, the first
comprehensive approach for incorporating various masking mechanisms into
Transformers architectures in a scalable way. We show that recent results on
linear causal attention (Choromanski et al., 2021) and log-linear RPE-attention
(Luo et al., 2021) are special cases of this general mechanism. However by
casting the problem as a topological (graph-based) modulation of unmasked
attention, we obtain several results unknown before, including efficient
d-dimensional RPE-masking and graph-kernel masking. We leverage many
mathematical techniques ranging from spectral analysis through dynamic
programming and random walks to new algorithms for solving Markov processes on
graphs. We provide a corresponding empirical evaluation.

26
The categories of this paper are: bayes-an
34
The categories of this paper are: chem-ph
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
20
The categories of this paper are: acc-phys
14
The categories of this paper are: math-ph
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
24
The categories of this paper are: dg-ga
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
33
The categories of this paper are: plasm-ph
32
The categories of this paper are: atom-ph
18
The categories of this paper are: econ