  Object detection is a fundamental task in computer vision, which has been
greatly progressed through developing large and intricate deep learning models.
However, the lack of transparency is a big challenge that may not allow the
widespread adoption of these models. Explainable artificial intelligence is a
field of research where methods are developed to help users understand the
behavior, decision logics, and vulnerabilities of AI-based systems. Black-box
explanation refers to explaining decisions of an AI system without having
access to its internals. In this paper, we design and implement a black-box
explanation method named Black-box Object Detection Explanation by Masking
(BODEM) through adopting a new masking approach for AI-based object detection
systems. We propose local and distant masking to generate multiple versions of
an input image. Local masks are used to disturb pixels within a target object
to figure out how the object detector reacts to these changes, while distant
masks are used to assess how the detection model's decisions are affected by
disturbing pixels outside the object. A saliency map is then created by
estimating the importance of pixels through measuring the difference between
the detection output before and after masking. Finally, a heatmap is created
that visualizes how important pixels within the input image are to the detected
objects. The experimentations on various object detection datasets and models
showed that BODEM can be effectively used to explain the behavior of object
detectors and reveal their vulnerabilities. This makes BODEM suitable for
explaining and validating AI based object detection systems in black-box
software testing scenarios. Furthermore, we conducted data augmentation
experiments that showed local masks produced by BODEM can be used for further
training the object detectors and improve their detection accuracy and
robustness.
