  Automatic dubbing, which generates a corresponding version of the input
speech in another language, could be widely utilized in many real-world
scenarios such as video and game localization. In addition to synthesizing the
translated scripts, automatic dubbing needs to further transfer the speaking
style in the original language to the dubbed speeches to give audiences the
impression that the characters are speaking in their native tongue. However,
state-of-the-art automatic dubbing systems only model the transfer on duration
and speaking rate, neglecting the other aspects in speaking style such as
emotion, intonation and emphasis which are also crucial to fully perform the
characters and speech understanding. In this paper, we propose a joint
multi-scale cross-lingual speaking style transfer framework to simultaneously
model the bidirectional speaking style transfer between languages at both
global (i.e. utterance level) and local (i.e. word level) scales. The global
and local speaking styles in each language are extracted and utilized to
predicted the global and local speaking styles in the other language with an
encoder-decoder framework for each direction and a shared bidirectional
attention mechanism for both directions. A multi-scale speaking style enhanced
FastSpeech 2 is then utilized to synthesize the predicted the global and local
speaking styles to speech for each language. Experiment results demonstrate the
effectiveness of our proposed framework, which outperforms a baseline with only
duration transfer in both objective and subjective evaluations.
