  Concept drift detection is crucial for many AI systems to ensure the system's
reliability. These systems often have to deal with large amounts of data or
react in real time. Thus, drift detectors must meet computational requirements
or constraints with a comprehensive performance evaluation. However, so far,
the focus of developing drift detectors is on detection quality, e.g.~accuracy,
but not on computational performance, such as running time. We show that the
previous works consider computational performance only as a secondary objective
and do not have a benchmark for such evaluation. Hence, we propose a set of
metrics that considers both, computational performance and detection quality.
Among others, our set of metrics includes the Relative Runtime Overhead RRO to
evaluate a drift detector's computational impact on an AI system. This work
focuses on unsupervised drift detectors, not being restricted to the
availability of labeled data. We measure the computational performance based on
the RRO and memory consumption of four available unsupervised drift detectors
on five different data sets. The range of the RRO reaches from 1.01 to 20.15.
Moreover, we measure state-of-the-art detection quality metrics to discuss our
evaluation results and show the necessity of thorough computational performance
considerations for drift detectors. Additionally, we highlight and explain
requirements for a comprehensive benchmark of drift detectors. Our
investigations can also be extended for supervised drift detection.
