  Neural Machine Translation (NMT) systems have received much recent attention
due to their human-level accuracy. While existing works mostly focus on either
improving accuracy or testing accuracy robustness, the computation efficiency
of NMT systems, which is of paramount importance due to often vast translation
demands and real-time requirements, has surprisingly received little attention.
In this paper, we make the first attempt to understand and test potential
computation efficiency robustness in state-of-the-art NMT systems. By analyzing
the working mechanism and implementation of 1455 public-accessible NMT systems,
we observe a fundamental property in NMT systems that could be manipulated in
an adversarial manner to reduce computation efficiency significantly. Our key
motivation is to generate test inputs that could sufficiently delay the
generation of EOS such that NMT systems would have to go through enough
iterations to satisfy the pre-configured threshold. We present NMTSloth, which
develops a gradient-guided technique that searches for a minimal and
unnoticeable perturbation at character-level, token-level, and structure-level,
which sufficiently delays the appearance of EOS and forces these inputs to
reach the naturally-unreachable threshold. To demonstrate the effectiveness of
NMTSloth, we conduct a systematic evaluation on three public-available NMT
systems: Google T5, AllenAI WMT14, and Helsinki-NLP translators. Experimental
results show that NMTSloth can increase NMT systems' response latency and
energy consumption by 85% to 3153% and 86% to 3052%, respectively, by
perturbing just one character or token in the input sentence. Our case study
shows that inputs generated by NMTSloth significantly affect the battery power
in real-world mobile devices (i.e., drain more than 30 times battery power than
normal inputs).
