  Embedded systems are ubiquitous. However, physical access of users and
likewise attackers makes them often threatened by fault attacks: a single fault
during the computation of a cryptographic primitive can lead to a total loss of
system security. This can have serious consequences, e.g., in safetycritical
systems, including bodily harm and catastrophic technical failures. However,
countermeasures often focus on isolated fault models and high layers of
abstraction. This leads to a dangerous sense of security, because exploitable
faults that are only visible at machine code level might not be covered by
countermeasures. In this work we present ARMORY, a fully automated open source
framework for exhaustive fault simulation on binaries of the ubiquitous ARM-M
class. It allows engineers and analysts to efficiently scan a binary for
potential weaknesses against arbitrary combinations of multi-variate fault
injections under a large variety of fault models. Using ARMORY, we demonstrate
the power of fully automated fault analysis and the dangerous implications of
applying countermeasures without knowledge of physical addresses and offsets.
We exemplarily analyze two case studies, which are highly relevant for
practice: a DFA on AES (cryptographic) and a secure bootloader
(non-cryptographic). Our results show that indeed numerous exploitable faults
found by ARMORY which occur in the actual implementations are easily missed in
manual inspection. Crucially, most faults are only visible when taking machine
code information, i.e., addresses and offsets, into account. Surprisingly, we
show that a countermeasure that protects against one type of fault can actually
largely increase the vulnerability to other fault models. Our work demonstrates
the need for countermeasures that, at least in their evaluation, are not
restricted to isolated fault models and consider low-level information [...].
