Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: CAPE: Encoding Relative Positions with Continuous Augmented Positional
  Embeddings   Without positional information, attention-based Transformer neural networks
are permutation-invariant. Absolute or relative positional embeddings are the
most popular ways to feed Transformer models with positional information.
Absolute positional embeddings are simple to implement, but suffer from
generalization issues when evaluating on sequences longer than seen at training
time. Relative positions are more robust to input length change, but are more
complex to implement and yield inferior model throughput due to extra
computational and memory costs. In this paper, we propose an augmentation-based
approach (CAPE) for absolute positional embeddings, which keeps the advantages
of both absolute (simplicity and speed) and relative positional embeddings
(better generalization). In addition, our empirical evaluation on
state-of-the-art models in machine translation, image and speech recognition
demonstrates that CAPE leads to better generalization performance as well as
increased stability with respect to training hyper-parameters.

34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
22
The categories of this paper are: chao-dyn
31
The categories of this paper are: ao-sci
17
The categories of this paper are: q-fin
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
18
The categories of this paper are: econ
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
20
The categories of this paper are: acc-phys
29
The categories of this paper are: funct-an
16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph