  Model compression via quantization and sparsity enhancement has gained an
immense interest to enable the deployment of deep neural networks (DNNs) in
resource-constrained edge environments. Although these techniques have shown
promising results in reducing the energy, latency and memory requirements of
the DNNs, their performance in non-ideal real-world settings (such as in the
presence of hardware faults) is yet to be completely understood. In this paper,
we investigate the impact of bit-flip and stuck-at faults on activation-sparse
quantized DNNs (QDNNs). We show that a high level of activation sparsity comes
at the cost of larger vulnerability to faults. For instance, activation-sparse
QDNNs exhibit up to 17.32% lower accuracy than the standard QDNNs. We also
establish that one of the major cause of the degraded accuracy is sharper
minima in the loss landscape for activation-sparse QDNNs, which makes them more
sensitive to perturbations in the weight values due to faults. Based on this
observation, we propose the mitigation of the impact of faults by employing a
sharpness-aware quantization (SAQ) training scheme. The activation-sparse and
standard QDNNs trained with SAQ have up to 36.71% and 24.76% higher inference
accuracy, respectively compared to their conventionally trained equivalents.
Moreover, we show that SAQ-trained activation-sparse QDNNs show better accuracy
in faulty settings than standard QDNNs trained conventionally. Thus the
proposed technique can be instrumental in achieving sparsity-related
energy/latency benefits without compromising on fault tolerance.
