  Soft errors in large VLSI circuits pose dramatic influence on computing- and
memory-intensive neural network (NN) processing. Understanding the influence of
soft errors on NNs is critical to protect against soft errors for reliable NN
processing. Prior work mainly rely on fault simulation to analyze the influence
of soft errors on NN processing. They are accurate but usually specific to
limited configurations of errors and NN models due to the prohibitively slow
simulation speed especially for large NN models and datasets. With the
observation that the influence of soft errors propagates across a large number
of neurons and accumulates as well, we propose to characterize the soft error
induced data disturbance on each neuron with normal distribution model
according to central limit theorem and develop a series of statistical models
to analyze the behavior of NN models under soft errors in general. The
statistical models reveal not only the correlation between soft errors and NN
model accuracy, but also how NN parameters such as quantization and
architecture affect the reliability of NNs. The proposed models are compared
with fault simulation and verified comprehensively. In addition, we observe
that the statistical models that characterize the soft error influence can also
be utilized to predict fault simulation results in many cases and we explore
the use of the proposed statistical models to accelerate fault simulations of
NNs. According to our experiments, the accelerated fault simulation shows
almost two orders of magnitude speedup with negligible simulation accuracy loss
over the baseline fault simulations.
