  Contextual knowledge is essential for reducing speech recognition errors on
high-valued long-tail words. This paper proposes a novel tree-constrained
pointer generator (TCPGen) component that enables end-to-end ASR models to bias
towards a list of long-tail words obtained using external contextual
information. With only a small overhead in memory use and computation cost,
TCPGen can structure thousands of biasing words efficiently into a symbolic
prefix-tree and creates a neural shortcut between the tree and the final ASR
output to facilitate the recognition of the biasing words. To enhance TCPGen,
we further propose a novel minimum biasing word error (MBWE) loss that directly
optimises biasing word errors during training, along with a biasing-word-driven
language model discounting (BLMD) method during the test. All contextual ASR
systems were evaluated on the public Librispeech audiobook corpus and the data
from the dialogue state tracking challenges (DSTC) with the biasing lists
extracted from the dialogue-system ontology. Consistent word error rate (WER)
reductions were achieved with TCPGen, which were particularly significant on
the biasing words with around 40\% relative reductions in the recognition error
rates. MBWE and BLMD further improved the effectiveness of TCPGen and achieved
more significant WER reductions on the biasing words. TCPGen also achieved
zero-shot learning of words not in the audio training set with large WER
reductions on the out-of-vocabulary words in the biasing list.
