  We present WinSyn, a dataset consisting of high-resolution photographs and
renderings of 3D models as a testbed for synthetic-to-real research. The
dataset consists of 75,739 high-resolution photographs of building windows,
including traditional and modern designs, captured globally. These include
89,318 cropped subimages of windows, of which 9,002 are semantically labeled.
Further, we present our domain-matched photorealistic procedural model which
enables experimentation over a variety of parameter distributions and
engineering approaches. Our procedural model provides a second corresponding
dataset of 21,290 synthetic images. This jointly developed dataset is designed
to facilitate research in the field of synthetic-to-real learning and synthetic
data generation. WinSyn allows experimentation into the factors that make it
challenging for synthetic data to compete with real-world data. We perform
ablations using our synthetic model to identify the salient rendering,
materials, and geometric factors pertinent to accuracy within the labeling
task. We chose windows as a benchmark because they exhibit a large variability
of geometry and materials in their design, making them ideal to study synthetic
data generation in a constrained setting. We argue that the dataset is a
crucial step to enable future research in synthetic data generation for deep
learning.
