Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Improving Transformer-based Speech Recognition Using Unsupervised
  Pre-training   Speech recognition technologies are gaining enormous popularity in various
industrial applications. However, building a good speech recognition system
usually requires large amounts of transcribed data, which is expensive to
collect. To tackle this problem, an unsupervised pre-training method called
Masked Predictive Coding is proposed, which can be applied for unsupervised
pre-training with Transformer based model. Experiments on HKUST show that using
the same training data, we can achieve CER 23.3%, exceeding the best end-to-end
model by over 0.2% absolute CER. With more pre-training data, we can further
reduce the CER to 21.0%, or a 11.8% relative CER reduction over baseline.

15
The categories of this paper are: nucl-ex
27
The categories of this paper are: comp-gas
5
The categories of this paper are: gr-qc
13
The categories of this paper are: quant-ph
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
9
The categories of this paper are: nlin
26
The categories of this paper are: bayes-an
3
The categories of this paper are: physics
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
30
The categories of this paper are: q-alg
36
The categories of this paper are: cmp-lg