Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Kernel Deformed Exponential Families for Sparse Continuous Attention   Attention mechanisms take an expectation of a data representation with
respect to probability weights. This creates summary statistics that focus on
important features. Recently, (Martins et al. 2020, 2021) proposed continuous
attention mechanisms, focusing on unimodal attention densities from the
exponential and deformed exponential families: the latter has sparse support.
(Farinhas et al. 2021) extended this to use Gaussian mixture attention
densities, which are a flexible class with dense support. In this paper, we
extend this to two general flexible classes: kernel exponential families and
our new sparse counterpart kernel deformed exponential families. Theoretically,
we show new existence results for both kernel exponential and deformed
exponential families, and that the deformed case has similar approximation
capabilities to kernel exponential families. Experiments show that kernel
deformed exponential families can attend to multiple compact regions of the
data domain.

17
The categories of this paper are: q-fin
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
30
The categories of this paper are: q-alg
18
The categories of this paper are: econ
2
The categories of this paper are: math
27
The categories of this paper are: comp-gas
8
The categories of this paper are: hep-ex
3
The categories of this paper are: physics
19
The categories of this paper are: eess
29
The categories of this paper are: funct-an
0
The categories of this paper are: hep-ph
31
The categories of this paper are: ao-sci