  Spiking neural networks (SNNs) have manifested remarkable advantages in power
consumption and event-driven property during the inference process. To take
full advantage of low power consumption and improve the efficiency of these
models further, the pruning methods have been explored to find sparse SNNs
without redundancy connections after training. However, parameter redundancy
still hinders the efficiency of SNNs during training. In the human brain, the
rewiring process of neural networks is highly dynamic, while synaptic
connections maintain relatively sparse during brain development. Inspired by
this, here we propose an efficient evolutionary structure learning (ESL)
framework for SNNs, named ESL-SNNs, to implement the sparse SNN training from
scratch. The pruning and regeneration of synaptic connections in SNNs evolve
dynamically during learning, yet keep the structural sparsity at a certain
level. As a result, the ESL-SNNs can search for optimal sparse connectivity by
exploring all possible parameters across time. Our experiments show that the
proposed ESL-SNNs framework is able to learn SNNs with sparse structures
effectively while reducing the limited accuracy. The ESL-SNNs achieve merely
0.28% accuracy loss with 10% connection density on the DVS-Cifar10 dataset. Our
work presents a brand-new approach for sparse training of SNNs from scratch
with biologically plausible evolutionary mechanisms, closing the gap in the
expressibility between sparse training and dense training. Hence, it has great
potential for SNN lightweight training and inference with low power consumption
and small memory usage.
