  AI models have shown promise in many medical imaging tasks. However, our
ability to explain what signals these models have learned is severely lacking.
Explanations are needed in order to increase the trust in AI-based models, and
could enable novel scientific discovery by uncovering signals in the data that
are not yet known to experts. In this paper, we present a method for automatic
visual explanations leveraging team-based expertise by generating hypotheses of
what visual signals in the images are correlated with the task. We propose the
following 4 steps: (i) Train a classifier to perform a given task (ii) Train a
classifier guided StyleGAN-based image generator (StylEx) (iii) Automatically
detect and visualize the top visual attributes that the classifier is sensitive
towards (iv) Formulate hypotheses for the underlying mechanisms, to stimulate
future research. Specifically, we present the discovered attributes to an
interdisciplinary panel of experts so that hypotheses can account for social
and structural determinants of health. We demonstrate results on eight
prediction tasks across three medical imaging modalities: retinal fundus
photographs, external eye photographs, and chest radiographs. We showcase
examples of attributes that capture clinically known features, confounders that
arise from factors beyond physiological mechanisms, and reveal a number of
physiologically plausible novel attributes. Our approach has the potential to
enable researchers to better understand, improve their assessment, and extract
new knowledge from AI-based models. Importantly, we highlight that attributes
generated by our framework can capture phenomena beyond physiology or
pathophysiology, reflecting the real world nature of healthcare delivery and
socio-cultural factors. Finally, we intend to release code to enable
researchers to train their own StylEx models and analyze their predictive
tasks.
