  Face recognition algorithms based on deep convolutional neural networks
(DCNNs) have made progress on the task of recognizing faces in unconstrained
viewing conditions. These networks operate with compact feature-based face
representations derived from learning a very large number of face images. While
the learned features produced by DCNNs can be highly robust to changes in
viewpoint, illumination, and appearance, little is known about the nature of
the face code that emerges at the top level of such networks. We analyzed the
DCNN features produced by two face recognition algorithms. In the first set of
experiments we used the top-level features from the DCNNs as input into linear
classifiers aimed at predicting metadata about the images. The results show
that the DCNN features contain surprisingly accurate information about the yaw
and pitch of a face, and about whether the face came from a still image or a
video frame. In the second set of experiments, we measured the extent to which
individual DCNN features operated in a view-dependent or view-invariant manner.
We found that view-dependent coding was a characteristic of the identities
rather than the DCNN features - with some identities coded consistently in a
view-dependent way and others in a view-independent way. In our third analysis,
we visualized the DCNN feature space for over 24,000 images of 500 identities.
Images in the center of the space were uniformly of low quality (e.g., extreme
views, face occlusion, low resolution). Image quality increased monotonically
as a function of distance from the origin. This result suggests that image
quality information is available in the DCNN features, such that consistently
average feature values reflect coding failures that reliably indicate poor or
unusable images. Combined, the results offer insight into the coding mechanisms
that support robust representation of faces in DCNNs.
