  This paper presents an accented text-to-speech (TTS) synthesis framework with
limited training data. We study two aspects concerning accent rendering:
phonetic (phoneme difference) and prosodic (pitch pattern and phoneme duration)
variations. The proposed accented TTS framework consists of two models: an
accented front-end for grapheme-to-phoneme (G2P) conversion and an accented
acoustic model with integrated pitch and duration predictors for
phoneme-to-Mel-spectrogram prediction. The accented front-end directly models
the phonetic variation, while the accented acoustic model explicitly controls
the prosodic variation. Specifically, both models are first pre-trained on a
large amount of data, then only the accent-related layers are fine-tuned on a
limited amount of data for the target accent. In the experiments, speech data
of three English accents, i.e., General American English, Irish English, and
British English Received Pronunciation, are used for pre-training. The
pre-trained models are then fine-tuned with Scottish and General Australian
English accents, respectively. Both objective and subjective evaluation results
show that the accented TTS front-end fine-tuned with a small accented phonetic
lexicon (5k words) effectively handles the phonetic variation of accents, while
the accented TTS acoustic model fine-tuned with a limited amount of accented
speech data (approximately 3 minutes) effectively improves the prosodic
rendering including pitch and duration. The overall accent modeling contributes
to improved speech quality and accent similarity.
