  The increased utilization of Artificial Intelligence (AI) solutions brings
with it inherent risks, such as misclassification and sub-optimal execution
time performance, due to errors introduced in their deployment infrastructure
because of problematic configuration and software faults. On top of that, AI
methods such as Deep Neural Networks (DNNs) are utilized to perform demanding,
resource-intensive and even safety-critical tasks, and in order to effectively
increase the performance of the DNN models deployed, a variety of Machine
Learning (ML) compilers have been developed, allowing compatibility of DNNs
with a variety of hardware acceleration devices, such as GPUs and TPUs.
Furthermore the correctness of the compilation process should be verified. In
order to allow developers and researchers to explore the robustness of DNN
models deployed on different hardware accelerators via ML compilers, in this
paper we propose MutateNN, a tool that provides mutation testing and model
analysis features in the context of deployment on different hardware
accelerators. To demonstrate the capabilities of MutateNN, we focus on the
image recognition domain by applying mutation testing to 7 well-established
models utilized for image classification. We instruct 21 mutations of 6
different categories, and deploy our mutants on 4 different hardware
acceleration devices of varying capabilities. Our results indicate that models
are proven robust to changes related to layer modifications and arithmetic
operators, while presenting discrepancies of up to 90.3% in mutants related to
conditional operators. We also observed unexpectedly severe performance
degradation on mutations related to arithmetic types of variables, leading the
mutants to produce the same classifications for all dataset inputs.
