  We introduce the first learning-based reconstructability predictor to improve
view and path planning for large-scale 3D urban scene acquisition using
unmanned drones. In contrast to previous heuristic approaches, our method
learns a model that explicitly predicts how well a 3D urban scene will be
reconstructed from a set of viewpoints. To make such a model trainable and
simultaneously applicable to drone path planning, we simulate the proxy-based
3D scene reconstruction during training to set up the prediction. Specifically,
the neural network we design is trained to predict the scene reconstructability
as a function of the proxy geometry, a set of viewpoints, and optionally a
series of scene images acquired in flight. To reconstruct a new urban scene, we
first build the 3D scene proxy, then rely on the predicted reconstruction
quality and uncertainty measures by our network, based off of the proxy
geometry, to guide the drone path planning. We demonstrate that our data-driven
reconstructability predictions are more closely correlated to the true
reconstruction quality than prior heuristic measures. Further, our learned
predictor can be easily integrated into existing path planners to yield
improvements. Finally, we devise a new iterative view planning framework, based
on the learned reconstructability, and show superior performance of the new
planner when reconstructing both synthetic and real scenes.
