Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for
  Low-resource Speech Recognition   End-to-end models have achieved impressive results on the task of automatic
speech recognition (ASR). For low-resource ASR tasks, however, labeled data can
hardly satisfy the demand of end-to-end models. Self-supervised acoustic
pre-training has already shown its amazing ASR performance, while the
transcription is still inadequate for language modeling in end-to-end models.
In this work, we fuse a pre-trained acoustic encoder (wav2vec2.0) and a
pre-trained linguistic encoder (BERT) into an end-to-end ASR model. The fused
model only needs to learn the transfer from speech to language during
fine-tuning on limited labeled data. The length of the two modalities is
matched by a monotonic attention mechanism without additional parameters.
Besides, a fully connected layer is introduced for the hidden mapping between
modalities. We further propose a scheduled fine-tuning strategy to preserve and
utilize the text context modeling ability of the pre-trained linguistic
encoder. Experiments show our effective utilizing of pre-trained modules. Our
model achieves better recognition performance on CALLHOME corpus (15 hours)
than other end-to-end models.

11
The categories of this paper are: hep-lat
8
The categories of this paper are: hep-ex
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
20
The categories of this paper are: acc-phys
7
The categories of this paper are: hep-th
15
The categories of this paper are: nucl-ex
0
The categories of this paper are: hep-ph
2
The categories of this paper are: math
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
22
The categories of this paper are: chao-dyn
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph