Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Sequence-to-Sequence Learning via Attention Transfer for Incremental
  Speech Recognition   Attention-based sequence-to-sequence automatic speech recognition (ASR)
requires a significant delay to recognize long utterances because the output is
generated after receiving entire input sequences. Although several studies
recently proposed sequence mechanisms for incremental speech recognition (ISR),
using different frameworks and learning algorithms is more complicated than the
standard ASR model. One main reason is because the model needs to decide the
incremental steps and learn the transcription that aligns with the current
short speech segment. In this work, we investigate whether it is possible to
employ the original architecture of attention-based ASR for ISR tasks by
treating a full-utterance ASR as the teacher model and the ISR as the student
model. We design an alternative student network that, instead of using a
thinner or a shallower model, keeps the original architecture of the teacher
model but with shorter sequences (few encoder and decoder states). Using
attention transfer, the student network learns to mimic the same alignment
between the current input short speech segments and the transcription. Our
experiments show that by delaying the starting time of recognition process with
about 1.7 sec, we can achieve comparable performance to one that needs to wait
until the end.

22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
7
The categories of this paper are: hep-th
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph
9
The categories of this paper are: nlin
6
The categories of this paper are: astro-ph
35
The categories of this paper are: mtrl-th
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin