  We propose a novel ECGAN for the challenging semantic image synthesis task.
Although considerable improvement has been achieved, the quality of synthesized
images is far from satisfactory due to three largely unresolved challenges. 1)
The semantic labels do not provide detailed structural information, making it
difficult to synthesize local details and structures. 2) The widely adopted CNN
operations such as convolution, down-sampling, and normalization usually cause
spatial resolution loss and thus cannot fully preserve the original semantic
information, leading to semantically inconsistent results. 3) Existing semantic
image synthesis methods focus on modeling local semantic information from a
single input semantic layout. However, they ignore global semantic information
of multiple input semantic layouts, i.e., semantic cross-relations between
pixels across different input layouts. To tackle 1), we propose to use edge as
an intermediate representation which is further adopted to guide image
generation via a proposed attention guided edge transfer module. Edge
information is produced by a convolutional generator and introduces detailed
structure information. To tackle 2), we design an effective module to
selectively highlight class-dependent feature maps according to the original
semantic layout to preserve the semantic information. To tackle 3), inspired by
current methods in contrastive learning, we propose a novel contrastive
learning method, which aims to enforce pixel embeddings belonging to the same
semantic class to generate more similar image content than those from different
classes. Doing so can capture more semantic relations by explicitly exploring
the structures of labeled pixels from multiple input semantic layouts.
Experiments on three challenging datasets show that our ECGAN achieves
significantly better results than state-of-the-art methods.
