  Over the recent years, a significant number of complex, deep neural networks
have been developed for a variety of applications including speech and face
recognition, computer vision in the areas of health-care, automatic
translation, image classification, etc. Moreover, there is an increasing demand
in deploying these networks in resource-constrained edge devices. As the
computational demands of these models keep increasing, pushing to their limits
the targeted devices, the constant development of new hardware systems tailored
to those workloads has been observed. Since programmability of these diverse
and complex platforms -- compounded by the rapid development of new DNN models
-- is a major challenge, platform vendors have developed Machine Learning
tailored SDKs to maximize the platform's performance.
  This work investigates the performance achieved on a number of modern
commodity embedded platforms coupled with the vendors' provided software
support when state-of-the-art DNN models from image classification, object
detection and image segmentation are targeted. The work quantifies the relative
latency gains of the particular embedded platforms and provides insights on the
relationship between the required minimum batch size for achieving maximum
throughput, concluding that modern embedded systems reach their maximum
performance even for modest batch sizes when a modern state of the art DNN
model is targeted. Overall, the presented results provide a guide for the
expected performance for a number of state-of-the-art DNNs on popular embedded
platforms across the image classification, detection and segmentation domains.
