Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Statistical Testing on ASR Performance via Blockwise Bootstrap   A common question being raised in automatic speech recognition (ASR)
evaluations is how reliable is an observed word error rate (WER) improvement
comparing two ASR systems, where statistical hypothesis testing and confidence
interval (CI) can be utilized to tell whether this improvement is real or only
due to random chance. The bootstrap resampling method has been popular for such
significance analysis which is intuitive and easy to use. However, this method
fails in dealing with dependent data, which is prevalent in speech world - for
example, ASR performance on utterances from the same speaker could be
correlated. In this paper we present blockwise bootstrap approach - by dividing
evaluation utterances into nonoverlapping blocks, this method resamples these
blocks instead of original data. We show that the resulting variance estimator
of absolute WER difference between two ASR systems is consistent under mild
conditions. We also demonstrate the validity of blockwise bootstrap method on
both synthetic and real-world speech data.

22
The categories of this paper are: chao-dyn
26
The categories of this paper are: bayes-an
9
The categories of this paper are: nlin
25
The categories of this paper are: patt-sol
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph
35
The categories of this paper are: mtrl-th
15
The categories of this paper are: nucl-ex
33
The categories of this paper are: plasm-ph
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
5
The categories of this paper are: gr-qc
13
The categories of this paper are: quant-ph