text: We introduce a novel strategy for learning to extract semantically meaningful features from aerial imagery. Instead of manually labeling the aerial imagery, we propose to predict (noisy) semantic features automatically extracted from co-located ground imagery. Our network architecture takes an aerial image as input, extracts features using a convolutional neural network, and then applies an adaptive transformation to map these features into the ground-level perspective. We use an end-to-end learning approach to minimize the difference between the semantic segmentation extracted directly from the ground image and the semantic segmentation predicted solely based on the aerial image. We show that a model learned using this strategy, with no additional training, is already capable of rough semantic labeling of aerial imagery. Furthermore, we demonstrate that by finetuning this model we can achieve more accurate semantic segmentation than two baseline initialization strategies. We use our network to address the task of estimating the geolocation and geoorientation of a ground image. Finally, we show how features extracted from an aerial image can be used to hallucinate a plausible ground-level panorama.
  @cite_18: Abstract Detecting building structures in aerial images is a task of importance for many applications. Low-level segmentation rarely gives a complete outline of the desired structures. We use a generic model of the shapes of the structures we are looking for — that they are rectangular or composed of rectangular components. We also use shadows cast by buildings to confirm their presence and to estimate their height. Our techniques have been tested on images with density typical of suburban areas.
  @cite_28: When training a system to label images, the amount of labeled training data tends to be a limiting factor. We consider the task of learning to label aerial images from existing maps. These provide abundant labels, but the labels are often incomplete and sometimes poorly registered. We propose two robust loss functions for dealing with these kinds of label noise and use the loss functions to train a deep neural network on two challenging aerial image datasets. The robust loss functions lead to big improvements in performance and our best system substantially outperforms the best published results on the task we consider.
  @cite_19: Abstract Object detection in optical remote sensing images, being a fundamental but challenging problem in the field of aerial and satellite image analysis, plays an important role for a wide range of applications and is receiving significant attention in recent years. While enormous methods exist, a deep review of the literature concerning generic object detection is still lacking. This paper aims to provide a review of the recent progress in this field. Different from several previously published surveys that focus on a specific object class such as building and road, we concentrate on more generic object categories including, but are not limited to, road, building, tree, vehicle, ship, airport, urban-area. Covering about 270 publications we survey (1) template matching-based object detection methods, (2) knowledge-based object detection methods, (3) object-based image analysis (OBIA)-based object detection methods, (4) machine learning-based object detection methods, and (5) five publicly available datasets and three standard evaluation metrics. We also discuss the challenges of current studies and propose two promising research directions, namely deep learning-based feature representation and weakly supervised learning-based geospatial object detection. It is our hope that this survey will be beneficial for the researchers to have better understanding of this research field.
  @cite_16: We present a system for the reconstruction of houses from aerial images. Most of the methods proposed so far use rather specific models. These models do not hold when dealing with Europeanised houses. Our system works with rules instead of models merely based on shape. This enables us to include additional domain specific knowledge, allowing a larger variability of the objects. However this also necessitates a more complex structure of the knowledge base and a more sophisticated reasoning control. We explain the practical application of such a system for the reconstruction of a house from an aerial image.
  @cite_13: 
  @cite_20: Reliably extracting information from aerial imagery is a difficult problem with many practical applications. One specific case of this problem is the task of automatically detecting roads. This task is a difficult vision problem because of occlusions, shadows, and a wide variety of non-road objects. Despite 30 years of work on automatic road detection, no automatic or semi-automatic road detection system is currently on the market and no published method has been shown to work reliably on large datasets of urban imagery. We propose detecting roads using a neural network with millions of trainable weights which looks at a much larger context than was used in previous attempts at learning the task. The network is trained on massive amounts of data using a consumer GPU. We demonstrate that predictive performance can be substantially improved by initializing the feature detectors using recently developed unsupervised learning methods as well as by taking advantage of the local spatial coherence of the output labels.We show that our method works reliably on two challenging urban datasets that are an order of magnitude larger than what was used to evaluate previous approaches.
summary: There is a long tradition of using computer vision techniques for aerial and satellite image understanding @cite_18 @cite_16 @cite_19 . Historically these two domains were distinct. Satellite imagery was typically lower-resolution, from a strictly top-down view, and with a diversity of spectral bands. Aerial imagery was typically higher-resolution, with a greater diversity of viewing angles, but with only RGB and NIR sensors. Recently these two domains have converged; we will use the term aerial imagery as we are primarily working with high-resolution RGB imagery. However, our approach could be applied to many types of aerial and satellite imagery. Kluckner al @cite_13 address the task of semantic segmentation using a random forest to combine color and height information. More recent work has explored the use of CNNs for aerial image understanding. Mnih and Hinton propose a CNN for detecting roads in aerial imagery @cite_20 using GIS data as ground truth. They extend their approach to handle omission noise and misregistration between the imagery and the labels @cite_28 . These approaches require either extensive pixel-level manual annotation or existing GIS data. Our work is the first to demonstrate the ability to transfer a dense pixel-level labeling of ground imagery to aerial imagery.