text: Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism into a computational model remains elusive. Encouraged by a new discovery (CVPR 2016) demonstrating extremely high inter-person accuracy of human perceived symmetries in the wild, we have created the first deep-learning neural network for reflection and rotation symmetry detection (Sym-NET), trained on photos from MS-COCO (Common Object in COntext) dataset with nearly 11K symmetry-labels from more than 400 human observers. We employ novel methods to convert discrete human labels into symmetry heatmaps, capture symmetry densely in an image and quantitatively evaluate Sym-NET against multiple existing computer vision algorithms. Using the symmetry competition testsets from CVPR 2013 and unseen MS-COCO photos, Sym-NET comes out as the winner with significantly superior performance over all other competitors. Beyond mathematically well-defined symmetries on a plane, Sym-NET demonstrates abilities to identify viewpoint-varied 3D symmetries, partially occluded symmetrical objects and symmetries at a semantic level.
  @cite_69: 
  @cite_73: Abstract This paper proposes an artificial neural network that extracts axes of symmetry from visual patterns. The input patterns can be plane figures, complicated line drawings or gray-scaled natural images taken by CCD cameras. The network has a hierarchical multi-layered architecture, which resembles that of the lower stages of the neocognitron. It consists of a contrast-extracting layer, edge-extracting layers (simple and complex types), and layers extracting symmetry axes. The network extracts oriented edges from the input image first, and then tries to extract axes of symmetry. Our network checks conditions of symmetry, not directly from the oriented edges, but from a blurred version of them. The use of blurred signals not only reduces the computational cost greatly, but also endows the network with a large tolerance to deformation of input patterns. It is important to get blurred signals, not directly from an input image, but from the oriented edges. Although information of edge locations becomes ambiguous after the blurring operation, most of important features of the original image can still remain stable. If the input image is directly blurred, however, most of the important features in the image will be lost.
  @cite_41: This paper shows that the introduction of non-uniform blur is very useful for comparing images, and proposes a neural network model that extracts axes of symmetry from visual patterns. The blurring operation greatly increases robustness against deformations and various kinds of noise, and largely reduces computational cost. Asymmetry between two groups of signals can be detected in a single action by the use of non-uniform blur having a cone-shaped distribution.The proposed model is a hierarchical multi-layered network, which consists of a contrast-extracting layer, edge-extracting layers (simple and complex types), and layers extracting symmetry axes. The model extracts oriented edges from an input image first, and then tries to extract axes of symmetry. The model checks conditions of symmetry, not directly from the oriented edges, but from a blurred version of the response of edge-extracting layer. The input patterns can be complicated line drawings, plane figures or gray-scaled natural images taken by CCD cameras.
  @cite_9: In this work we propose a learning-based approach to symmetry detection in natural images. We focus on ribbon-like structures, i.e. contours marking local and approximate reflection symmetry and make three contributions to improve their detection. First, we create and make publicly available a ground-truth dataset for this task by building on the Berkeley Segmentation Dataset. Second, we extract features representing multiple complementary cues, such as grayscale structure, color, texture, and spectral clustering information. Third, we use supervised learning to learn how to combine these cues, and employ MIL to accommodate the unknown scale and orientation of the symmetric structures. We systematically evaluate the performance contribution of each individual component in our pipeline, and demonstrate that overall we consistently improve upon results obtained using existing alternatives.
  @cite_6: The role of symmetry in computer vision has waxed and waned in importance during the evolution of the field from its earliest days. At first figuring prominently in support of bottom-up indexing, it fell out of favour as shape gave way to appearance and recognition gave way to detection. With a strong prior in the form of a target object, the role of the weaker priors offered by perceptual grouping was greatly diminished. However, as the field returns to the problem of recognition from a large database, the bottom-up recovery of the parts that make up the objects in a cluttered scene is critical for their recognition. The medial axis community has long exploited the ubiquitous regularity of symmetry as a basis for the decomposition of a closed contour into medial parts. However, today’s recognition systems are faced with cluttered scenes and the assumption that a closed contour exists, i.e., that figure-ground segmentation has been solved, rendering much of the medial axis community’s work inapplicable. In this article, we review a computational framework, previously reported in [1–3], that bridges the representation power of the medial axis and the need to recover and group an object’s parts in a cluttered scene. Our framework is rooted in the idea that a maximally-inscribed disc, the building block of a medial axis, can be modelled as a compact superpixel in the image. We evaluate the method on images of cluttered scenes.
  @cite_34: We present two methods for detecting symmetry in images, one based directly on the intensity values and another one based on a discrete representation of local orientation. A symmetry finder has been developed which uses the intensity-based method to search an image for compact regions which display some degree of mirror symmetry due to intensity similarities across a straight axis. In a different approach, we look at symmetry as a bilateral relationship between local orientations. A symmetryenhancing edge detector is presented which indicates edges dependent on the orientations at two different image positions. SEED, as we call it, is a detector element implemented by a feedforward network that holds the symmetry conditions. We use SEED to find the contours of symmetric objects of which we know the axis of symmetry from the intensity-based symmetry finder. The methods presented have been applied to the problem of visually guided car-following. Real-time experiments with a system for automatic headway control on motorways have been successful.
  @cite_66: Object skeleton is a useful cue for object detection, complementary to the object contour, as it provides a structural representation to describe the relationship among object parts. While object skeleton extraction in natural images is a very challenging problem, as it requires the extractor to be able to capture both local and global image context to determine the intrinsic scale of each skeleton pixel. Existing methods rely on per-pixel based multi-scale feature computation, which results in difficult modeling and high time consumption. In this paper, we present a fully convolutional network with multiple scale-associated side outputs to address this problem. By observing the relationship between the receptive field sizes of the sequential stages in the network and the skeleton scales they can capture, we introduce a scale-associated side output to each stage. We impose supervision to different stages by guiding the scale-associated side outputs toward groundtruth skeletons of different scales. The responses of the multiple scale-associated side outputs are then fused in a scale-specific way to localize skeleton pixels with multiple scales effectively. Our method achieves promising results on two skeleton extraction datasets, and significantly outperforms other competitors.
  @cite_20: Local reflection symmetry detection in nature images is a quite important but challenging task in computer vision. The main obstacle is both the scales and the orientations of symmetric structure are unknown. The multiple instance learning (MIL) framework sheds lights onto this task owing to its capability to well accommodate the unknown scales and orientations of the symmetric structures. However, to differentiate symmetry vs non-symmetry remains to face extreme confusions caused by clutters scenes and ambiguous object structures. In this paper, we propose a novel multiple instance learning framework for local reflection symmetry detection, named multiple instance subspace learning (MISL), which instead learns a group of models respectively on well partitioned subspaces. To obtain such subspaces, we propose an efficient dividing strategy under MIL setting, named partial random projection tree (PRPT), by taking advantage of the fact that each sample (bag) is represented by the proposed symmetry features computed at specific scale and orientation combinations (instances). Encouraging experimental results on two datasets demonstrate that the proposed local reflection symmetry detection method outperforms current state-of-the-arts. HighlightsWe perform clustering on samples represented by multiple instances.We learn a group of MIL classifiers on subspaces.We report state-of-the-arts results on the symmetry detection benchmark.
  @cite_17: 
summary: There have been some shallow-network reflection detection approaches (well before the current deep learning craze). Zielke al @cite_34 use a static feed forward method to enhance the symmetric edges for detection. The max operation between the different orientations is similar to other voting systems @cite_69 . Fukushima and Kikuchi @cite_41 @cite_73 present another neural network method for detecting reflection symmetry around the center of an image. They use a 4-layer network to find the symmetry axis. Skeletonization , a related problem to reflection detection, has attracted a lot of attention recently @cite_6 @cite_20 @cite_9 @cite_17 . Shen al @cite_66 use a deep CNN to learn symmetry at multiple scales and fuse the final output together. The network needs object skeleton ground-truth for the particular scale of the objects. The network outputs a skeleton heatmap which is thresholded to produce a binary image denoting the detected skeletons.