  The reliability evaluation of Deep Neural Networks (DNNs) executed on Graphic
Processing Units (GPUs) is a challenging problem since the hardware
architecture is highly complex and the software frameworks are composed of many
layers of abstraction. While software-level fault injection is a common and
fast way to evaluate the reliability of complex applications, it may produce
unrealistic results since it has limited access to the hardware resources and
the adopted fault models may be too naive (i.e., single and double bit flip).
Contrarily, physical fault injection with neutron beam provides realistic error
rates but lacks fault propagation visibility. This paper proposes a
characterization of the DNN fault model combining both neutron beam experiments
and fault injection at software level. We exposed GPUs running General Matrix
Multiplication (GEMM) and DNNs to beam neutrons to measure their error rate. On
DNNs, we observe that the percentage of critical errors can be up to 61%, and
show that ECC is ineffective in reducing critical errors. We then performed a
complementary software-level fault injection, using fault models derived from
RTL simulations. Our results show that by injecting complex fault models, the
YOLOv3 misdetection rate is validated to be very close to the rate measured
with beam experiments, which is 8.66x higher than the one measured with fault
injection using only single-bit flips.
