  In this paper we address the problem of unsupervised gaze correction in the
wild, presenting a solution that works without the need for precise annotations
of the gaze angle and the head pose. We have created a new dataset called
CelebAGaze, which consists of two domains X, Y, where the eyes are either
staring at the camera or somewhere else. Our method consists of three novel
modules: the Gaze Correction module (GCM), the Gaze Animation module (GAM), and
the Pretrained Autoencoder module (PAM). Specifically, GCM and GAM separately
train a dual in-painting network using data from the domain $X$ for gaze
correction and data from the domain $Y$ for gaze animation. Additionally, a
Synthesis-As-Training method is proposed when training GAM to encourage the
features encoded from the eye region to be correlated with the angle
information, resulting in a gaze animation which can be achieved by
interpolation in the latent space. To further preserve the identity
information~(e.g., eye shape, iris color), we propose the PAM with an
Autoencoder, which is based on Self-Supervised mirror learning where the
bottleneck features are angle-invariant and which works as an extra input to
the dual in-painting models. Extensive experiments validate the effectiveness
of the proposed method for gaze correction and gaze animation in the wild and
demonstrate the superiority of our approach in producing more compelling
results than state-of-the-art baselines. Our code, the pretrained models and
the supplementary material are available at:
https://github.com/zhangqianhui/GazeAnimation.
