  We introduce beta diffusion, a novel generative modeling method that
integrates demasking and denoising to generate data within bounded ranges.
Using scaled and shifted beta distributions, beta diffusion utilizes
multiplicative transitions over time to create both forward and reverse
diffusion processes, maintaining beta distributions in both the forward
marginals and the reverse conditionals, given the data at any point in time.
Unlike traditional diffusion-based generative models relying on additive
Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is
multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived
from the convexity of the KL divergence. We demonstrate that the proposed KLUBs
are more effective for optimizing beta diffusion compared to negative ELBOs,
which can also be derived as the KLUBs of the same KL divergence with its two
arguments swapped. The loss function of beta diffusion, expressed in terms of
Bregman divergence, further supports the efficacy of KLUBs for optimization.
Experimental results on both synthetic data and natural images demonstrate the
unique capabilities of beta diffusion in generative modeling of range-bounded
data and validate the effectiveness of KLUBs in optimizing diffusion models,
thereby making them valuable additions to the family of diffusion-based
generative models and the optimization techniques used to train them.
