  Virtual try-on is a promising application of computer graphics and human
computer interaction that can have a profound real-world impact especially
during this pandemic. Existing image-based works try to synthesize a try-on
image from a single image of a target garment, but it inherently limits the
ability to react to possible interactions. It is difficult to reproduce the
change of wrinkles caused by pose and body size change, as well as pulling and
stretching of the garment by hand. In this paper, we propose an alternative per
garment capture and synthesis workflow to handle such rich interactions by
training the model with many systematically captured images. Our workflow is
composed of two parts: garment capturing and clothed person image synthesis. We
designed an actuated mannequin and an efficient capturing process that collects
the detailed deformations of the target garments under diverse body sizes and
poses. Furthermore, we proposed to use a custom-designed measurement garment,
and we captured paired images of the measurement garment and the target
garments. We then learn a mapping between the measurement garment and the
target garments using deep image-to-image translation. The customer can then
try on the target garments interactively during online shopping.
