Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Soft Truncation: A Universal Training Technique of Score-based Diffusion
  Model for High Precision Score Estimation   Recent advances in diffusion models bring the state-of-the art performance on
image generation tasks. However, empirical results on previous research in
diffusion models imply that there is an inverse correlation on performances for
density estimation and sample generation. This paper analyzes that the inverse
correlation arises because density estimation is mostly contributed from small
diffusion time, whereas sample generation mainly depends on large diffusion
time. However, training score network on both small and large diffusion time is
demanding because of the loss imbalance issue. To successfully train the score
network on both small and large diffusion time, this paper introduces a
training technique, Soft Truncation, that softens the truncation time for every
mini-batch update, which is universally applicable to any types of diffusion
models. It turns out that Soft Truncation is equivalent to a diffusion model
with a general weight, and we prove the variational bound of the general
weighted diffusion model. In view of this variational bound, Soft Truncation
becomes a natural way to train the score network. In experiments, Soft
Truncation achieves the state-of-the-art performance on CIFAR-10, CelebA,
CelebA-HQ $256\times 256$, and STL-10 datasets.

0
The categories of this paper are: hep-ph
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
29
The categories of this paper are: funct-an
18
The categories of this paper are: econ
35
The categories of this paper are: mtrl-th
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
4
The categories of this paper are: cond-mat
11
The categories of this paper are: hep-lat
6
The categories of this paper are: astro-ph
21
The categories of this paper are: adap-org
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin
19
The categories of this paper are: eess
32
The categories of this paper are: atom-ph