  Deep Learning (DL) systems have proliferated in many applications, requiring
specialized hardware accelerators and chips. In the nano-era, devices have
become increasingly more susceptible to permanent and transient faults.
Therefore, we need an efficient methodology for analyzing the resilience of
advanced DL systems against such faults, and understand how the faults in
neural accelerator chips manifest as errors at the DL application level, where
faults can lead to undetectable and unrecoverable errors. Using fault
injection, we can perform resilience investigations of the DL system by
modifying neuron weights and outputs at the software-level, as if the hardware
had been affected by a transient fault. Existing fault models reduce the search
space, allowing faster analysis, but requiring a-priori knowledge on the model,
and not allowing further analysis of the filtered-out search space. Therefore,
we propose ISimDL, a novel methodology that employs neuron sensitivity to
generate importance sampling-based fault-scenarios. Without any a-priori
knowledge of the model-under-test, ISimDL provides an equivalent reduction of
the search space as existing works, while allowing long simulations to cover
all the possible faults, improving on existing model requirements. Our
experiments show that the importance sampling provides up to 15x higher
precision in selecting critical faults than the random uniform sampling,
reaching such precision in less than 100 faults. Additionally, we showcase
another practical use-case for importance sampling for reliable DNN design,
namely Fault Aware Training (FAT). By using ISimDL to select the faults leading
to errors, we can insert the faults during the DNN training process to harden
the DNN against such faults. Using importance sampling in FAT reduces the
overhead required for finding faults that lead to a predetermined drop in
accuracy by more than 12x.
