  Spiking Neural Network (SNN) is known as the most famous brain-inspired
model, but the non-differentiable spiking mechanism makes it hard to train
large-scale SNNs. To facilitate the training of large-scale SNNs, many training
methods are borrowed from Artificial Neural Networks (ANNs), among which deep
residual learning is the most commonly used. But the unique features of SNNs
make prior intuition built upon ANNs not available for SNNs. Although there are
a few studies that have made some pioneer attempts on the topology of Spiking
ResNet, the advantages of different connections remain unclear. To tackle this
issue, we analyze the merits and limitations of various residual connections
and empirically demonstrate our ideas with extensive experiments. Then, based
on our observations, we abstract the best-performing connections into densely
additive (DA) connection, extend such a concept to other topologies, and
propose four architectures for training large-scale SNNs, termed DANet, which
brings up to 13.24% accuracy gain on ImageNet. Besides, in order to present a
detailed methodology for designing the topology of large-scale SNNs, we further
conduct in-depth discussions on their applicable scenarios in terms of their
performance on various scales of datasets and demonstrate their advantages over
prior architectures. At a low training expense, our best-performing
ResNet-50/101/152 obtain 73.71%/76.13%/77.22% top-1 accuracy on ImageNet with 4
time steps. We believe that this work shall give more insights for future works
to design the topology of their networks and promote the development of
large-scale SNNs. The code will be publicly available.
