Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis
  via Non-Autoregressive Generative Transformers   Conditional image synthesis aims to create an image according to some
multi-modal guidance in the forms of textual descriptions, reference images,
and image blocks to preserve, as well as their combinations. In this paper,
instead of investigating these control signals separately, we propose a new
two-stage architecture, M6-UFC, to unify any number of multi-modal controls. In
M6-UFC, both the diverse control signals and the synthesized image are
uniformly represented as a sequence of discrete tokens to be processed by
Transformer. Different from existing two-stage autoregressive approaches such
as DALL-E and VQGAN, M6-UFC adopts non-autoregressive generation (NAR) at the
second stage to enhance the holistic consistency of the synthesized image, to
support preserving specified image blocks, and to improve the synthesis speed.
Further, we design a progressive algorithm that iteratively improves the
non-autoregressively generated image, with the help of two estimators developed
for evaluating the compliance with the controls and evaluating the fidelity of
the synthesized image, respectively. Extensive experiments on a newly collected
large-scale clothing dataset M2C-Fashion and a facial dataset Multi-Modal
CelebA-HQ verify that M6-UFC can synthesize high-fidelity images that comply
with flexible multi-modal controls.

20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
29
The categories of this paper are: funct-an
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
17
The categories of this paper are: q-fin
30
The categories of this paper are: q-alg
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math
34
The categories of this paper are: chem-ph
22
The categories of this paper are: chao-dyn
36
The categories of this paper are: cmp-lg
32
The categories of this paper are: atom-ph
19
The categories of this paper are: eess
35
The categories of this paper are: mtrl-th