  Machine learning methods for conditional data generation usually build a
mapping from source conditional data X to target data Y. The target Y (e.g.,
text, speech, music, image, video) is usually high-dimensional and complex, and
contains information that does not exist in source data, which hinders
effective and efficient learning on the source-target mapping. In this paper,
we present a learning paradigm called regeneration learning for data
generation, which first generates Y' (an abstraction/representation of Y) from
X and then generates Y from Y'. During training, Y' is obtained from Y through
either handcrafted rules or self-supervised learning and is used to learn
X-->Y' and Y'-->Y. Regeneration learning extends the concept of representation
learning to data generation tasks, and can be regarded as a counterpart of
traditional representation learning, since 1) regeneration learning handles the
abstraction (Y') of the target data Y for data generation while traditional
representation learning handles the abstraction (X') of source data X for data
understanding; 2) both the processes of Y'-->Y in regeneration learning and
X-->X' in representation learning can be learned in a self-supervised way
(e.g., pre-training); 3) both the mappings from X to Y' in regeneration
learning and from X' to Y in representation learning are simpler than the
direct mapping from X to Y. We show that regeneration learning can be a
widely-used paradigm for data generation (e.g., text generation, speech
recognition, speech synthesis, music composition, image generation, and video
generation) and can provide valuable insights into developing data generation
methods.
