  Deep neural networks (DNNs) are found to be vulnerable to adversarial
attacks, and various methods have been proposed for the defense. Among these
methods, adversarial training has been drawing increasing attention because of
its simplicity and effectiveness. However, the performance of the adversarial
training is greatly limited by the architectures of target DNNs, which often
makes the resulting DNNs with poor accuracy and unsatisfactory robustness. To
address this problem, we propose DSARA to automatically search for the neural
architectures that are accurate and robust after adversarial training. In
particular, we design a novel cell-based search space specially for adversarial
training, which improves the accuracy and the robustness upper bound of the
searched architectures by carefully designing the placement of the cells and
the proportional relationship of the filter numbers. Then we propose a
two-stage search strategy to search for both accurate and robust neural
architectures. At the first stage, the architecture parameters are optimized to
minimize the adversarial loss, which makes full use of the effectiveness of the
adversarial training in enhancing the robustness. At the second stage, the
architecture parameters are optimized to minimize both the natural loss and the
adversarial loss utilizing the proposed multi-objective adversarial training
method, so that the searched neural architectures are both accurate and robust.
We evaluate the proposed algorithm under natural data and various adversarial
attacks, which reveals the superiority of the proposed method in terms of both
accurate and robust architectures. We also conclude that accurate and robust
neural architectures tend to deploy very different structures near the input
and the output, which has great practical significance on both hand-crafting
and automatically designing of accurate and robust neural architectures.
