  Machine learning (ML) models that learn and predict properties of computer
programs are increasingly being adopted and deployed. These models have
demonstrated success in applications such as auto-completing code, summarizing
large programs, and detecting bugs and malware in programs. In this work, we
investigate principled ways to adversarially perturb a computer program to fool
such learned models, and thus determine their adversarial robustness. We use
program obfuscations, which have conventionally been used to avoid attempts at
reverse engineering programs, as adversarial perturbations. These perturbations
modify programs in ways that do not alter their functionality but can be
crafted to deceive an ML model when making a decision. We provide a general
formulation for an adversarial program that allows applying multiple
obfuscation transformations to a program in any language. We develop
first-order optimization algorithms to efficiently determine two key aspects --
which parts of the program to transform, and what transformations to use. We
show that it is important to optimize both these aspects to generate the best
adversarially perturbed program. Due to the discrete nature of this problem, we
also propose using randomized smoothing to improve the attack loss landscape to
ease optimization. We evaluate our work on Python and Java programs on the
problem of program summarization. We show that our best attack proposal
achieves a $52\%$ improvement over a state-of-the-art attack generation
approach for programs trained on a seq2seq model. We further show that our
formulation is better at training models that are robust to adversarial
attacks.
