  Novel view synthesis has recently been revolutionized by learning neural
radiance fields directly from sparse observations. However, rendering images
with this new paradigm is slow due to the fact that an accurate quadrature of
the volume rendering equation requires a large number of samples for each ray.
Previous work has mainly focused on speeding up the network evaluations that
are associated with each sample point, e.g., via caching of radiance values
into explicit spatial data structures, but this comes at the expense of model
compactness. In this paper, we propose a novel dual-network architecture that
takes an orthogonal direction by learning how to best reduce the number of
required sample points. To this end, we split our network into a sampling and
shading network that are jointly trained. Our training scheme employs fixed
sample positions along each ray, and incrementally introduces sparsity
throughout training to achieve high quality even at low sample counts. After
fine-tuning with the target number of samples, the resulting compact neural
representation can be rendered in real-time. Our experiments demonstrate that
our approach outperforms concurrent compact neural representations in terms of
quality and frame rate and performs on par with highly efficient hybrid
representations. Code and supplementary material is available at
https://thomasneff.github.io/adanerf.
