  Recent advancements in neural end-to-end TTS models have shown high-quality,
natural synthesized speech in a conventional sentence-based TTS. However, it is
still challenging to reproduce similar high quality when a whole paragraph is
considered in TTS, where a large amount of contextual information needs to be
considered in building a paragraph-based TTS model. To alleviate the difficulty
in training, we propose to model linguistic and prosodic information by
considering cross-sentence, embedded structure in training. Three sub-modules,
including linguistics-aware, prosody-aware and sentence-position networks, are
trained together with a modified Tacotron2. Specifically, to learn the
information embedded in a paragraph and the relations among the corresponding
component sentences, we utilize linguistics-aware and prosody-aware networks.
The information in a paragraph is captured by encoders and the inter-sentence
information in a paragraph is learned with multi-head attention mechanisms. The
relative sentence position in a paragraph is explicitly exploited by a
sentence-position network. Trained on a storytelling audio-book corpus (4.08
hours), recorded by a female Mandarin Chinese speaker, the proposed TTS model
demonstrates that it can produce rather natural and good-quality speech
paragraph-wise. The cross-sentence contextual information, such as break and
prosodic variations between consecutive sentences, can be better predicted and
rendered than the sentence-based model. Tested on paragraph texts, of which the
lengths are similar to, longer than, or much longer than the typical paragraph
length of the training data, the TTS speech produced by the new model is
consistently preferred over the sentence-based model in subjective tests and
confirmed in objective measures.
