text: Recently, spiking neural network (SNN) has received significant attentions for its biological plausibility. SNN theoretically has at least the same computational power as traditional artificial neural networks (ANNs), and it has the potential to achieve revolutionary energy-efficiency. However, at current stage, it is still a big challenge to train a very deep SNN. In this paper, we propose an efficient approach to build a spiking version of deep residual network (ResNet), which represents the state-of-the-art convolutional neural networks (CNNs). We employ the idea of converting a trained ResNet to a network of spiking neurons named Spiking ResNet. To address the conversion problem, we propose a shortcut normalisation mechanism to appropriately scale continuous-valued activations to match firing rates in SNN, and a layer-wise error compensation approach to reduce the error caused by discretisation. Experimental results on MNIST, CIFAR-10, and CIFAR-100 demonstrate that the proposed Spiking ResNet yields the state-of-the-art performance of SNNs.
  @cite_30: 
  @cite_26: In cultures of dissociated rat hippocampal neurons, persistent potentiation and depression of glutamatergic synapses were induced by correlated spiking of presynaptic and postsynaptic neurons. The relative timing between the presynaptic and postsynaptic spiking determined the direction and the extent of synaptic changes. Repetitive postsynaptic spiking within a time window of 20 msec after presynaptic activation resulted in long-term potentiation (LTP), whereas postsynaptic spiking within a window of 20 msec before the repetitive presynaptic activation led to long-term depression (LTD). Significant LTP occurred only at synapses with relatively low initial strength, whereas the extent of LTD did not show obvious dependence on the initial synaptic strength. Both LTP and LTD depended on the activation of NMDA receptors and were absent in cases in which the postsynaptic neurons were GABAergic in nature. Blockade of L-type calcium channels with nimodipine abolished the induction of LTD and reduced the extent of LTP. These results underscore the importance of precise spike timing, synaptic strength, and postsynaptic cell type in the activity-induced modification of central synapses and suggest that Hebb’s rule may need to incorporate a quantitative consideration of spike timing that reflects the narrow and asymmetric window for the induction of synaptic modification.
  @cite_7: Memristive nanodevices can feature a compact multilevel nonvolatile memory function, but are prone to device variability. We propose a novel neural network-based computing paradigm, which exploits their specific physics, and which has virtual immunity to their variability. Memristive devices are used as synapses in a spiking neural network performing unsupervised learning. They learn using a simplified and customized “spike timing dependent plasticity” rule. In the network, neurons’ threshold is adjusted following a homeostasis-type rule. We perform system level simulations with an experimentally verified model of the memristive devices’ behavior. They show, on the textbook case of character recognition, that performance can compare with traditional supervised networks of similar complexity. They also show that the system can retain functionality with extreme variations of various memristive devices’ parameters (a relative standard dispersion of more than 50 is tolerated on all device parameters), thanks to the robustness of the scheme, its unsupervised nature, and the capability of homeostasis. Additionally the network can adjust to stimuli presented with different coding schemes, is particularly robust to read disturb effects and does not require unrealistic control on the devices’ conductance. These results open the way for a novel design approach for ultraadaptive electronic systems.
  @cite_23: This paper focuses on feedforward spiking neuron models of the visual cortex. Essentially, we show that a combination of a temporal coding scheme where the most strongly activated neurons fire first with Spike Timing-Dependent Plasticity leads to a situation where neurons will gradually become selective to visual patterns that are both salient, and consistently present in the inputs. At the same time, their responses become more and more rapid. These responses can then be used very effectively to perform object recognition in natural images.
  @cite_12: Spike timing dependent plasticity (STDP) is a learning rule that modifies synaptic strength as a function of the relative timing of pre- and postsynaptic spikes. When a neuron is repeatedly presented with similar inputs, STDP is known to have the effect of concentrating high synaptic weights on afferents that systematically fire early, while postsynaptic spike latencies decrease. Here we use this learning rule in an asynchronous feedforward spiking neural network that mimics the ventral visual pathway and shows that when the network is presented with natural images, selectivity to intermediate-complexity visual features emerges. Those features, which correspond to prototypical patterns that are both salient and consistently present in the images, are highly informative and enable robust object recognition, as demonstrated on various classification tasks. Taken together, these results show that temporal codes may be a key to understanding the phenomenal processing speed achieved by the visual system and that STDP can lead to fast and selective responses.
summary: Another approach is to learn with spike-timing-dependent plasticity (STDP) @cite_26 , a biologically plausible approach. In @cite_12 @cite_23 , STDP has been proven able to select features in an unsupervised manner. Improved STDP algorithms such as rectangular STDP @cite_7 and exponential STDP @cite_30 can achieve competitive performance on MNIST dataset with a simple two-layer network.