text: Deep neural networks trained over large datasets learn features that are both generic to the whole dataset, and specific to individual classes in the dataset. Learned features tend towards generic in the lower layers and specific in the higher layers of a network. Methods like fine-tuning are made possible because of the ability for one filter to apply to multiple target classes. Much like the human brain this behavior, can also be used to cluster and separate classes. However, to the best of our knowledge there is no metric for how applicable learned features are to specific classes. In this paper we propose a definition and metric for measuring the applicability of learned features to individual classes, and use this applicability metric to estimate input applicability and produce a new method of unsupervised learning we call the CactusNet.
  @cite_10: A Fuzzy Adaptive Resonance Theory (ART) model capable of rapid stable learning of recognition categories in response to arbitrary sequences of analog or binary input patterns is described. Fuzzy ART incorporates computations from fuzzy set theory into the ART 1 neural network, which learns to categorize only binary input patterns. The generalization to learning both analog and binary input patterns is achieved by replacing appearances of the intersection operator ( N ) in ART 1 by the MIN operator ( ) of fuzzy set theory. The MIN operator reduces to the intersection operator in the binarv case. Category proliferation is prevented by, normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the M1N operator ( ) and the MAX operator (V) of fuzzy set theory play complementary roles. Complement coding uses on-cells and of cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of categorv "boxes." Smaller vigilance values lead to larger category boxes. Learning stops when the input space is covered by boxes. With fast learning and a finite input set of arbitrary size and composition, learning stabilizes after just one presentation of each input pattern. A fast-commit slow-recode option combines fast learning with a forgetting rule that buffers system memory against noise. Using this option, rare events can be rapidly learned, yet previously learned memories are not rapidly erased in response to statistically unreliable input fluctuations.
  @cite_18: SyNAPSE program of the Defense Advanced Projects Research Agency (Hewlett-Packard Company, subcontract under DARPA prime contract HR0011-09-3-0001, and HRL Laboratories LLC, subcontract #801881-BS under DARPA prime contract HR0011-09-C-0001); CELEST, an NSF Science of Learning Center (SBE-0354378)
  @cite_1: Adaptive Resonance Theory, or ART, is a cognitive and neural theory of how the brain autonomously learns to categorize, recognize, and predict objects and events in a changing world. This article reviews classical and recent developments of ART, and provides a synthesis of concepts, principles, mechanisms, architectures, and the interdisciplinary data bases that they have helped to explain and predict. The review illustrates that ART is currently the most highly developed cognitive and neural theory available, with the broadest explanatory and predictive range. Central to ART's predictive power is its ability to carry out fast, incremental, and stable unsupervised and supervised learning in response to a changing world. ART specifies mechanistic links between processes of consciousness, learning, expectation, attention, resonance, and synchrony during both unsupervised and supervised learning. ART provides functional and mechanistic explanations of such diverse topics as laminar cortical circuitry; invariant object and scenic gist learning and recognition; prototype, surface, and boundary attention; gamma and beta oscillations; learning of entorhinal grid cells and hippocampal place cells; computation of homologous spatial and temporal mechanisms in the entorhinal-hippocampal system; vigilance breakdowns during autism and medial temporal amnesia; cognitive-emotional interactions that focus attention on valued objects in an adaptively timed way; item-order-rank working memories and learned list chunks for the planning and control of sequences of linguistic, spatial, and motor information; conscious speech percepts that are influenced by future context; auditory streaming in noise during source segregation; and speaker normalization. Brain regions that are functionally described include visual and auditory neocortex; specific and nonspecific thalamic nuclei; inferotemporal, pariet al, prefrontal, entorhinal, hippocampal, parahippocampal, perirhinal, and motor cortices; frontal eye fields; supplementary eye fields; amygdala; basal ganglia: cerebellum; and superior colliculus. Due to the complementary organization of the brain, ART does not describe many spatial and motor behaviors whose matching and learning laws differ from those of ART. ART algorithms for engineering and technology are listed, as are comparisons with other types of models.
  @cite_23: Functional ond mechanistic comparisons are mode between several network models of cognitive processing: competitive learning, interactive activation, adaptive resonance, and back propagation. The starting point of this comparison is the article of Rumelhart ond Zipser (1985) on feature discovery through competitive learning. All the models which Rumelhart and Zipser (1985) have described were shown in Grossberg (1976b) to exhibit a type of learning which is temporally unstable. Competitive learning mechanisms con be stabilized in response to an arbitrary input environment by being supplemented with mechanisms for learning top-down expectancies, or templates; for matching bottom-up input patterns with the top-down expectancies; and for releasing orienting reactions in o mismatch situation, thereby updating short-term memory ond searching for another internal representation. Network architectures which embody all of these mechonisms were called adaptive resonance models by Grossberg (1976 ). Self-stabilizing learning models are candidates for use in real-world applications where unpredictable changes can occur in complex input environments. Competitive learning postulates ore inconsistent with the postulates of the interactive activotion model of McClelland and Rumelhart (1981). and suggest different levels of processing and interaction rules for the analysis of word recognition. Adaptive resonance models use these alternative levels and interaction rules. The selforganizing learning of on odaptive resonance model is compared ond contrasted with the teacher-directed learning of a back propagation model. A number of criteria for evaluating reol-time network models of cognitive processing ore described and applied.
summary: The human mind identifies and clusters objects based on their features regardless of whether an object is known or not @cite_1 . Adaptive resonance theory (ART) @cite_18 @cite_23 is a machine learning theory that attempts to determine whether an object belongs to a known object class by comparing the detected features of the object with the expected features of all known classes individually. If the smallest difference between the detected features of the object and some known class's expected features is within a set threshold then the object is classified and is considered to belong to that class. This threshold is known as the vigilance parameter. If the difference exceeds the vigilance parameter, however, the object is considered to belong to a new class. This allows ART to perform unsupervised learning as it classifies not based on a target class, but differences in features. Over the years, several new variations of ART have been proposed including Fuzzy ART @cite_10 which, uses fuzzy logic to improve ART's stability.