  Movie productions use high resolution 3d characters with complex proprietary
rigs to create the highest quality images possible for large displays.
Unfortunately, these 3d assets are typically not compatible with real-time
graphics engines used for games, mixed reality and real-time pre-visualization.
Consequently, the 3d characters need to be re-modeled and re-rigged for these
new applications, requiring weeks of work and artistic approval. Our solution
to this problem is to learn a compact image-based rendering of the original 3d
character, conditioned directly on the rig parameters. Our idea is to render
the character in many different poses and views, and to train a deep neural
network to render high resolution images, from the rig parameters directly.
Many neural rendering techniques have been proposed to render from 2d
skeletons, or geometry and UV maps. However these require manual work, and to
do not remain compatible with the animator workflow of manipulating rig
widgets, as well as the real-time game engine pipeline of interpolating rig
parameters. We extend our architecture to support dynamic re-lighting and
composition with other 3d objects in the scene. We designed a network that
efficiently generates multiple scene feature maps such as normals, depth,
albedo and mask, which are composed with other scene objects to form the final
image.
