  Deep Learning (DL) is increasingly used in safety-critical applications,
raising concerns about its reliability. DL suffers from a well-known problem of
lacking robustness, especially when faced with adversarial perturbations known
as Adversarial Examples (AEs). Despite recent efforts to detect AEs using
advanced attack and testing methods, these approaches often overlook the input
distribution and perceptual quality of the perturbations. As a result, the
detected AEs may not be relevant in practical applications or may appear
unrealistic to human observers. This can waste testing resources on rare AEs
that seldom occur during real-world use, limiting improvements in DL model
dependability.
  In this paper, we propose a new robustness testing approach for detecting AEs
that considers both the feature level distribution and the pixel level
distribution, capturing the perceptual quality of adversarial perturbations.
The two considerations are encoded by a novel hierarchical mechanism. First, we
select test seeds based on the density of feature level distribution and the
vulnerability of adversarial robustness. The vulnerability of test seeds are
indicated by the auxiliary information, that are highly correlated with local
robustness. Given a test seed, we then develop a novel genetic algorithm based
local test case generation method, in which two fitness functions work
alternatively to control the perceptual quality of detected AEs. Finally,
extensive experiments confirm that our holistic approach considering
hierarchical distributions is superior to the state-of-the-arts that either
disregard any input distribution or only consider a single (non-hierarchical)
distribution, in terms of not only detecting imperceptible AEs but also
improving the overall robustness of the DL model under testing.
