text: Conventional Convolutional Neural Networks (CNNs) use either a linear or non-linear filter to extract features from an image patch (region) of spatial size @math (Typically, @math is small and is equal to @math , e.g., @math is 5 or 7). Generally, the size of the filter is equal to the size @math of the input patch. We argue that the representation ability of equal-size strategy is not strong enough. To overcome the drawback, we propose to use subpatch filter whose spatial size @math is smaller than @math . The proposed subpatch filter consists of two subsequent filters. The first one is a linear filter of spatial size @math and is aimed at extracting features from spatial domain. The second one is of spatial size @math and is used for strengthening the connection between different input feature channels and for reducing the number of parameters. The subpatch filter convolves with the input patch and the resulting network is called a subpatch network. Taking the output of one subpatch network as input, we further repeat constructing subpatch networks until the output contains only one neuron in spatial domain. These subpatch networks form a new network called Cascaded Subpatch Network (CSNet). The feature layer generated by CSNet is called csconv layer. For the whole input image, we construct a deep neural network by stacking a sequence of csconv layers. Experimental results on four benchmark datasets demonstrate the effectiveness and compactness of the proposed CSNet. For example, our CSNet reaches a test error of @math on the CIFAR10 dataset without model averaging. To the best of our knowledge, this is the best result ever obtained on the CIFAR10 dataset.
  @cite_34: 
  @cite_10: We present an integrated framework for using Convolutional Networks for classification, localization and detection. We show how a multiscale and sliding window approach can be efficiently implemented within a ConvNet. We also introduce a novel deep learning approach to localization by learning to predict object boundaries. Bounding boxes are then accumulated rather than suppressed in order to increase detection confidence. We show that different tasks can be learned simultaneously using a single shared network. This integrated framework is the winner of the localization task of the ImageNet Large Scale Visual Recognition Challenge 2013 (ILSVRC2013) and obtained very competitive results for the detection and classifications tasks. In post-competition work, we establish a new state of the art for the detection task. Finally, we release a feature extractor from our best model called OverFeat.
  @cite_6: Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.
summary: (1) . Some researchers paid attention to the parameters of CNN, such as the sizes of convolutional filters, the strides of filters, the number of feature channels in each layer, and the number of convolutional layers. They tried to adjust the parameters to improve the performance of CNN through exhaustive experiments. Zeiler and Fergus @cite_34 visualized the trained CNN model and found that large filter size and large stride of the first convolutional layer could cause aliasing artifacts. Therefore, they used smaller receptive window size and smaller stride. Sermanet @cite_10 utilized smaller strides in the first convolution, larger number of feature maps, and larger number of layers. They achieved better results than the AlexNet. The VGG network @cite_6 pushes the depth of CNN to up to 19 convolutional layers by using very small convolutional filters and gains a significant improvement. These above efforts can be viewed as preliminary explorations on how to construct networks with better performance.