  With the rise of AI in recent years and the increase in complexity of the
models, the growing demand in computational resources is starting to pose a
significant challenge. The need for higher compute power is being met with
increasingly more potent accelerators and the use of large compute clusters.
However, the gain in prediction accuracy from large models trained on
distributed and accelerated systems comes at the price of a substantial
increase in energy demand, and researchers have started questioning the
environmental friendliness of such AI methods at scale. Consequently, energy
efficiency plays an important role for AI model developers and infrastructure
operators alike. The energy consumption of AI workloads depends on the model
implementation and the utilized hardware. Therefore, accurate measurements of
the power draw of AI workflows on different types of compute nodes is key to
algorithmic improvements and the design of future compute clusters and
hardware. To this end, we present measurements of the energy consumption of two
typical applications of deep learning models on different types of compute
nodes. Our results indicate that 1. deriving energy consumption directly from
runtime is not accurate, but the consumption of the compute node needs to be
considered regarding its composition; 2. neglecting accelerator hardware on
mixed nodes results in overproportional inefficiency regarding energy
consumption; 3. energy consumption of model training and inference should be
considered separately - while training on GPUs outperforms all other node types
regarding both runtime and energy consumption, inference on CPU nodes can be
comparably efficient. One advantage of our approach is that the information on
energy consumption is available to all users of the supercomputer, enabling an
easy transfer to other workloads alongside a raise in user-awareness of energy
consumption.
