  Many high-performing machine learning models are not interpretable. As they
are increasingly used in decision scenarios that can critically affect
individuals, it is necessary to develop tools to better understand their
outputs. Popular explanation methods include contrastive explanations. However,
they suffer several shortcomings, among others an insufficient incorporation of
background knowledge, and a lack of interactivity. While (dialogue-like)
interactivity is important to better communicate an explanation, background
knowledge has the potential to significantly improve their quality, e.g., by
adapting the explanation to the needs of the end-user. To close this gap, we
present REASONX, an explanation tool based on Constraint Logic Programming
(CLP). REASONX provides interactive contrastive explanations that can be
augmented by background knowledge, and allows to operate under a setting of
under-specified information, leading to increased flexibility in the provided
explanations. REASONX computes factual and constrative decision rules, as well
as closest constrative examples. It provides explanations for decision trees,
which can be the ML models under analysis, or global/local surrogate models of
any ML model. While the core part of REASONX is built on CLP, we also provide a
program layer that allows to compute the explanations via Python, making the
tool accessible to a wider audience. We illustrate the capability of REASONX on
a synthetic data set, and on a a well-developed example in the credit domain.
In both cases, we can show how REASONX can be flexibly used and tailored to the
needs of the user.
