  Image recognition tasks typically use deep learning and require enormous
processing power, thus relying on hardware accelerators like GPUs and TPUs for
fast, timely processing. Failure in real-time image recognition tasks can occur
due to sub-optimal mapping on hardware accelerators during model deployment,
which may lead to timing uncertainty and erroneous behavior. Mapping on
hardware accelerators is done using multiple software components like deep
learning frameworks, compilers, and device libraries, that we refer to as the
computational environment. Owing to the increased use of image recognition
tasks in safety-critical applications like autonomous driving and medical
imaging, it is imperative to assess their robustness to changes in the
computational environment, as the impact of parameters like deep learning
frameworks, compiler optimizations, and hardware devices on model performance
and correctness is not yet well understood.
  In this paper we present a differential testing framework, DeltaNN, that
allows us to assess the impact of different computational environment
parameters on the performance of image recognition models during deployment,
post training. DeltaNN generates different implementations of a given image
recognition model for variations in environment parameters, namely, deep
learning frameworks, compiler optimizations and hardware devices and analyzes
differences in model performance as a result. Using DeltaNN, we conduct an
empirical study of robustness analysis of three popular image recognition
models using the ImageNet dataset. We report the impact in terms of
misclassifications and inference time differences across different settings. In
total, we observed up to 72% output label differences across deep learning
frameworks, and up to 81% unexpected performance degradation in terms of
inference time, when applying compiler optimizations.
