  This paper presents a process for estimating the spatially varying surface
reflectance of complex scenes observed under natural illumination. In contrast
to previous methods, our process is not limited to scenes viewed under
controlled lighting conditions but can handle complex indoor and outdoor scenes
viewed under arbitrary illumination conditions. An end-to-end process uses a
model of the scene's geometry and several images capturing the scene's surfaces
from arbitrary viewpoints and under various natural illumination conditions. We
develop a differentiable path tracer that leverages least-square conformal
mapping for handling multiple disjoint objects appearing in the scene. We
follow a two-step optimization process and introduce a multi-view gradient
consistency loss which results in up to 30-50% improvement in the image
reconstruction loss and can further achieve better disentanglement of the
diffuse and specular BRDFs compared to other state-of-the-art. We demonstrate
the process in real-world indoor and outdoor scenes from images in the wild and
show that we can produce realistic renders consistent with actual images using
the estimated reflectance properties. Experiments show that our technique
produces realistic results for arbitrary outdoor scenes with complex geometry.
The source code is publicly available at:
https://gitlab.com/alen.joy/multi-view-gradient-consistency-for-svbrdf-estimation-of-complex-scenes-under-natural-illumination
