Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Speechformer: Reducing Information Loss in Direct Speech Translation   Transformer-based models have gained increasing popularity achieving
state-of-the-art performance in many research fields including speech
translation. However, Transformer's quadratic complexity with respect to the
input sequence length prevents its adoption as is with audio signals, which are
typically represented by long sequences. Current solutions resort to an initial
sub-optimal compression based on a fixed sampling of raw audio features.
Therefore, potentially useful linguistic information is not accessible to
higher-level layers in the architecture. To solve this issue, we propose
Speechformer, an architecture that, thanks to reduced memory usage in the
attention layers, avoids the initial lossy compression and aggregates
information only at a higher level according to more informed linguistic
criteria. Experiments on three language pairs (en->de/es/nl) show the efficacy
of our solution, with gains of up to 0.8 BLEU on the standard MuST-C corpus and
of up to 4.0 BLEU in a low resource scenario.

31
The categories of this paper are: ao-sci
3
The categories of this paper are: physics
33
The categories of this paper are: plasm-ph
9
The categories of this paper are: nlin
7
The categories of this paper are: hep-th
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
27
The categories of this paper are: comp-gas
13
The categories of this paper are: quant-ph
11
The categories of this paper are: hep-lat
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
19
The categories of this paper are: eess
18
The categories of this paper are: econ
34
The categories of this paper are: chem-ph
22
The categories of this paper are: chao-dyn