Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Distilling Knowledge from Ensembles of Acoustic Models for Joint
  CTC-Attention End-to-End Speech Recognition   Knowledge distillation has been widely used to compress existing deep
learning models while preserving the performance on a wide range of
applications. In the specific context of Automatic Speech Recognition (ASR),
distillation from ensembles of acoustic models has recently shown promising
results in increasing recognition performance. In this paper, we propose an
extension of multi-teacher distillation methods to joint CTC-attention
end-to-end ASR systems. We also introduce three novel distillation strategies.
The core intuition behind them is to integrate the error rate metric to the
teacher selection rather than solely focusing on the observed losses. In this
way, we directly distill and optimize the student toward the relevant metric
for speech recognition. We evaluate these strategies under a selection of
training procedures on different datasets (TIMIT, Librispeech, Common Voice)
and various languages (English, French, Italian). In particular,
state-of-the-art error rates are reported on the Common Voice French, Italian
and TIMIT datasets.

12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
31
The categories of this paper are: ao-sci
11
The categories of this paper are: hep-lat
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
4
The categories of this paper are: cond-mat
0
The categories of this paper are: hep-ph
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
22
The categories of this paper are: chao-dyn
35
The categories of this paper are: mtrl-th