text: Appearance-based gaze estimation is believed to work well in real-world settings, but existing datasets have been collected under controlled laboratory conditions and methods have been not evaluated across multiple datasets. In this work we study appearance-based gaze estimation in the wild. We present the MPIIGaze dataset that contains 213,659 images we collected from 15 participants during natural everyday laptop use over more than three months. Our dataset is significantly more variable than existing ones with respect to appearance and illumination. We also present a method for in-the-wild appearance-based gaze estimation using multimodal convolutional neural networks that significantly outperforms state-of-the art methods in the most challenging cross-dataset evaluation. We present an extensive evaluation of several state-of-the-art image-based gaze estimation algorithms on three current datasets, including our own. This evaluation provides clear insights and allows us to identify key research challenges of gaze estimation in the wild.
  @cite_26: Eye contact plays a crucial role in our everyday social interactions. The ability of a device to reliably detect when a person is looking at it can lead to powerful human-object interfaces. Today, most gaze-based interactive systems rely on gaze tracking technology. Unfortunately, current gaze tracking techniques require active infrared illumination, calibration, or are sensitive to distance and pose. In this work, we propose a different solution-a passive, appearance-based approach for sensing eye contact in an image. By focusing on gaze *locking* rather than gaze tracking, we exploit the special appearance of direct eye gaze, achieving a Matthews correlation coefficient (MCC) of over 0.83 at long distances (up to 18 m) and large pose variations (up to ±30° of head yaw rotation) using a very basic classifier and without calibration. To train our detector, we also created a large publicly available gaze data set: 5,880 images of 56 people over varying gaze directions and head poses. We demonstrate how our method facilitates human-object interaction, user analytics, image filtering, and gaze-triggered photography.
  @cite_22: A multistage procedure to detect eye features is presented. Multiresolution and topographic classification are used to detect the iris center. The eye corner is calculated combining valley detection and eyelid curve extraction. The algorithm is tested in the BioID database and in a proprietary database containing more than 1200 images. The results show that the suggested algorithm is robust and accurate. Regarding the iris center our method obtains the best average behavior for the BioID database compared to other available algorithms. Additional contributions are that our algorithm functions in real time and does not require complex post processing stages.
  @cite_41: The lack of a common benchmark for the evaluation of the gaze estimation task from RGB and RGB-D data is a serious limitation for distinguishing the advantages and disadvantages of the many proposed algorithms found in the literature. This paper intends to overcome this limitation by introducing a novel database along with a common framework for the training and evaluation of gaze estimation approaches. In particular, we have designed this database to enable the evaluation of the robustness of algorithms with respect to the main challenges associated to this task: i) Head pose variations; ii) Person variation; iii) Changes in ambient and sensing conditions and iv) Types of target: screen or 3D object.
  @cite_2: Within the past decade, many computational approaches have been developed to estimate gaze directions of persons based on their facial appearance. Most researchers used common face datasets with only a l imited representation of different head poses to train and verify their algorithms. Moreover, in most datasets, faces have neither a defined gaze direction, nor do they incor porate different combinations of eye gaze and head pose. Therefore, we recorded an extended dataset of 20 sub jects including faces in various combinations of head pos e and eye gaze leading to a total amount of 2220 colour i mages (111 per subject). The images were produced under controlled conditions, i.e. we used a technique to make sure that the subjects adjust their head and eyes appropriately. Furthermore, all images are manually labelled with landmarks indicating important featur es of the face. Finally, we evaluate the dataset with two computational methods for head pose and gaze estimation respectively.
  @cite_16: This paper presents a new, publicly available eye tracking dataset, aimed to be used as a benchmark for Point of Gaze (PoG) detection algorithms. The dataset consists of a set of videos recording the eye motion of human test subjects as they were looking at, or following, a set of predefined points of interest on a computer visual display unit. The eye motion was recorded using a Mobile Eye, head mounted, infrared monocular camera. The ground truth of the point of gaze and head location and direction in the three dimensional space are provided together with the data. The ground truth regarding the point of gaze at is known in advance since the subjects are always looking at predefined targets, whereas, the head position in 3D is captured using a Vicon Motion Tracking System.
  @cite_25: Inferring human gaze from low-resolution eye images is still a challenging task despite its practical importance in many application scenarios. This paper presents a learning-by-synthesis approach to accurate image-based gaze estimation that is person- and head pose-independent. Unlike existing appearance-based methods that assume person-specific training data, we use a large amount of cross-subject training data to train a 3D gaze estimator. We collect the largest and fully calibrated multi-view gaze dataset and perform a 3D reconstruction in order to generate dense training data of eye images. By using the synthesized dataset to learn a random regression forest, we show that our method outperforms existing methods that use low-resolution eye images.
summary: Because most existing gaze estimation datasets are designed for coarse gaze estimation, the sampling density of gaze and head pose space is not sufficient to train appearance-based gaze estimators @cite_16 @cite_22 @cite_2 @cite_26 (see Table for an overview of existing datasets). More comparable to , the Eyediap dataset contains 94 video sequences of 16 participants looking at three different targets (discrete and continuous markers displayed on a monitor, and floating physical targets) under both static and free head motion @cite_41 . The UT Multiview dataset also contains dense gaze samples of 50 participants as well as 3D reconstructions of eye regions that can be used to synthesise images for arbitrary head poses @cite_25 . However, as discussed before, both datasets have the significant limitation that they were recorded under controlled laboratory settings. Although the Eyediap dataset includes two different illumination conditions, recordings under the second condition were provided only for a subset of the participants.