  We present the first neural relighting approach for rendering high-fidelity
personalized hands that can be animated in real-time under novel illumination.
Our approach adopts a teacher-student framework, where the teacher learns
appearance under a single point light from images captured in a light-stage,
allowing us to synthesize hands in arbitrary illuminations but with heavy
compute. Using images rendered by the teacher model as training data, an
efficient student model directly predicts appearance under natural
illuminations in real-time. To achieve generalization, we condition the student
model with physics-inspired illumination features such as visibility, diffuse
shading, and specular reflections computed on a coarse proxy geometry,
maintaining a small computational overhead. Our key insight is that these
features have strong correlation with subsequent global light transport
effects, which proves sufficient as conditioning data for the neural relighting
network. Moreover, in contrast to bottleneck illumination conditioning, these
features are spatially aligned based on underlying geometry, leading to better
generalization to unseen illuminations and poses. In our experiments, we
demonstrate the efficacy of our illumination feature representations,
outperforming baseline approaches. We also show that our approach can
photorealistically relight two interacting hands at real-time speeds.
https://sh8.io/#/relightable_hands
