  Sequence-to-Sequence (seq2seq) tasks transcribe the input sequence to a
target sequence. The Connectionist Temporal Classification (CTC) criterion is
widely used in multiple seq2seq tasks. Besides predicting the target sequence,
a side product of CTC is to predict the alignment, which is the most probable
input-long sequence that specifies a hard aligning relationship between the
input and target units. As there are multiple potential aligning sequences
(called paths) that are equally considered in CTC formulation, the choice of
which path will be most probable and become the predicted alignment is always
uncertain. In addition, it is usually observed that the alignment predicted by
vanilla CTC will drift compared with its reference and rarely provides
practical functionalities. Thus, the motivation of this work is to make the CTC
alignment prediction controllable and thus equip CTC with extra
functionalities. The Bayes risk CTC (BRCTC) criterion is then proposed in this
work, in which a customizable Bayes risk function is adopted to enforce the
desired characteristics of the predicted alignment. With the risk function, the
BRCTC is a general framework to adopt some customizable preference over the
paths in order to concentrate the posterior into a particular subset of the
paths. In applications, we explore one particular preference which yields
models with the down-sampling ability and reduced inference costs. By using
BRCTC with another preference for early emissions, we obtain an improved
performance-latency trade-off for online models. Experimentally, the proposed
BRCTC reduces the inference cost of offline models by up to 47% without
performance degradation and cuts down the overall latency of online systems to
an unseen level.
