text: Multispectral person detection aims at automatically localizing humans in images that consist of multiple spectral bands. Usually, the visual-optical (VIS) and the thermal infrared (IR) spectra are combined to achieve higher robustness for person detection especially in insufficiently illuminated scenes. This paper focuses on analyzing existing detection approaches for their generalization ability. Generalization is a key feature for machine learning based detection algorithms that are supposed to perform well across different datasets. Inspired by recent literature regarding person detection in the VIS spectrum, we perform a cross-validation study to empirically determine the most promising dataset to train a well-generalizing detector. Therefore, we pick one reference Deep Convolutional Neural Network (DCNN) architecture and three different multispectral datasets. The Region Proposal Network (RPN) originally introduced for object detection within the popular Faster R-CNN is chosen as a reference DCNN. The reason is that a stand-alone RPN is able to serve as a competitive detector for two-class problems such as person detection. Furthermore, current state-of-the-art approaches initially apply an RPN followed by individual classifiers. The three considered datasets are the KAIST Multispectral Pedestrian Benchmark including recently published improved annotations for training and testing, the Tokyo Multi-spectral Semantic Segmentation dataset, and the OSU Color-Thermal dataset including recently released annotations. The experimental results show that the KAIST Multispectral Pedestrian Benchmark with its improved annotations provides the best basis to train a DCNN with good generalization ability compared to the other two multispectral datasets. On average, this detection model achieves a log-average Miss Rate (MR) of 29.74 evaluated on the reasonable test subsets of the three datasets.
  @cite_30: 
  @cite_35: 
  @cite_4: Convnets have enabled significant progress in pedestrian detection recently, but there are still open questions regarding suitable architectures and training data. We revisit CNN design and point out key adaptations, enabling plain FasterRCNN to obtain state-of-the-art results on the Caltech dataset. To achieve further improvement from more and better data, we introduce CityPersons, a new set of person annotations on top of the Cityscapes dataset. The diversity of CityPersons allows us for the first time to train one single CNN model that generalizes well over multiple benchmarks. Moreover, with additional training with CityPersons, we obtain top results using FasterRCNN on Caltech, improving especially for more difficult cases (heavy occlusion and small scale) and providing higher localization quality.
  @cite_28: Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
  @cite_39: 
  @cite_0: Datasets are an integral part of contemporary object recognition research. They have been the chief reason for the considerable progress in the field, not just as source of large amounts of training data, but also as means of measuring and comparing performance of competing algorithms. At the same time, datasets have often been blamed for narrowing the focus of object recognition research, reducing it to a single benchmark performance number. Indeed, some datasets, that started out as data capture efforts aimed at representing the visual world, have become closed worlds unto themselves (e.g. the Corel world, the Caltech-101 world, the PASCAL VOC world). With the focus on beating the latest benchmark numbers on the latest dataset, have we perhaps lost sight of the original purpose? The goal of this paper is to take stock of the current state of recognition datasets. We present a comparison study using a set of popular datasets, evaluated based on a number of criteria including: relative data bias, cross-dataset generalization, effects of closed-world assumption, and sample value. The experimental results, some rather surprising, suggest directions that can improve dataset collection as well as algorithm evaluation protocols. But more broadly, the hope is to stimulate discussion in the community regarding this very important, but largely neglected issue.
  @cite_23: We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations. First, using three well known DNNs (ResNet-152, VGG-19, GoogLeNet) we find the human visual system to be more robust to nearly all of the tested image manipulations, and we observe progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker. Secondly, we show that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on, yet they display extremely poor generalisation abilities when tested on other distortion types. For example, training on salt-and-pepper noise does not imply robustness on uniform white noise and vice versa. Thus, changes in the noise distribution between training and testing constitutes a crucial challenge to deep learning vision systems that can be systematically addressed in a lifelong machine learning approach. Our new dataset consisting of 83K carefully measured human psychophysical trials provide a useful reference for lifelong robustness against image degradations set by the human visual system.
  @cite_2: Popular Abstract in Swedish Inom manga omraden, exempelvis inom sjukvarden, blir det allt vanligare med situationer dar stora mangder av bilder behover analyseras. Ofta kan dessa bildmangder vara sa stora att det tar for lang tid for en manniska att ga igenom dem. Till exempel kan man med hjalp av datortomografi ta tredimensionella rontgenbilder av manniskor, nagot som gors pa sjukhus idag. Att ga igenom dessa bilder manuellt tar givetvis massor av resurser fran lakare, resurser som skulle kunna anvandas battre. Ett av malen med datorseende ar att automatisera analysen av stora bildmangder. Ett av de forsta stegen nar man analyserar en bild automatiskt ar ofta att rita ut granser mellan olika objekt, att segmentera bilden i olika omraden. Nar man ska gora en sadan segmentering sa brukar man bygga upp en statistisk modell som sager att varje pixel i bilden med en viss sannolikhet tillhor ett visst objekt, exempelvis en lunga eller benmargen. Segmenteringen valjs sedan sa att varje pixel far tillhorighet till det objekt som har hogst sannolikhet i just den pixeln. Problemet ar dock att varken modellen eller bilderna ar perfekta och de resulterande segmenteringarna ar ofta brusiga. Ett satt att minska detta problem ar att infora mer regler i modellen. For de flesta bilder ar det till exempel troligt att pixlar som ligger nara varandra tillhor samma objekt. Att lagga in sadana regler i modellen brukar kallas att regularisera losningen. Regularisering kan aven anvandas vid automatisk generering av sa kallade djupkartor. I en djupkarta anges avstandet till varje objekt i en bild. Djupkartor ar exempelvis anvandbara inom spelindustrin. Med hjalp av djupkartor kan man till exempel mata en hands rorelser och pa sa sett anvanda kroppen som en handkontroll. I automatiskt genererade djupkartor finns ofta brus som man kan minska med hjalp av regularisering. En lagre ordningens regularisering innebar att man infor krav pa storheter som man kan uppskatta utifran tva punkter, exempelvis langden pa en kurva. I en hogre ordningens regularisering infor man krav pa storheter som kraver fler an tva punkter. Ett exempel pa en sadan storhet ar en kurvas krokning. Denna avhandling omfattar i huvudsak olika modeller for automatisk bildsegmentering och uppskattning av djupkartor. For de olika modellerna infors hogre ordningens regulariseringsmodeller och det visas hur man effektivt hittar bra losningar till dessa modeller. (Less)
  @cite_5: 
  @cite_10: Multispectral images that combine visual-optical (VIS) and infrared (IR) image information are a promising source of data for automatic person detection. Especially in automotive or surveillance applications, challenging conditions such as insufficient illumination or large distances between camera and object occur regularly and can affect image quality. This leads to weak image contrast or low object resolution. In order to detect persons under such conditions, we apply deep learning for effectively fusing the VIS and IR information in multispectral images. We present a novel multispectral Region Proposal Network (RPN) that is built up on the pre-trained very deep convolutional network VGG-16. The proposals of this network are further evaluated using a Boosted Decision Trees classifier in order to reduce potential false positive detections. With a log-average miss rate of 29:83 on the reasonable test set of the KAIST Multispectral Pedestrian Detection Benchmark, we improve the current state-of-the-art by about 18 .
  @cite_12: 
  @cite_11: Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9 top-5 validation error (and 4.8 test error), exceeding the accuracy of human raters.
summary: Generalization in computer vision and machine learning is the ability of image and video processing algorithms such as object detectors or classifiers to perform well not only on the dataset they are trained on but also on previously unseen data acquired by different cameras or under different environmental conditions such as daytime vs. nighttime. Usually, such datasets underlie different image quality perturbations such as specific sensor noise, contrast, or blur that can severely affect the processing quality @cite_23 . Detector or classifier models exclusively and extensively trained on this data are prone to overfit @cite_30 or getting biased @cite_0 . In contrast to explicit methods that aim at reducing or avoiding such effects during model training @cite_28 @cite_11 @cite_39 @cite_12 or application @cite_2 , we are inspired by Torralba and Efros @cite_0 and @cite_4 and compare models trained on different datasets in a cross-validation manner. In addition, different datasets for domain adaptation and transfer learning @cite_5 @cite_10 @cite_35 are exploited that best support the multispectral fusion.