  Topological data analysis (TDA) is a branch of computational mathematics,
bridging algebraic topology and data science, that provides compact,
noise-robust representations of complex structures. Deep neural networks (DNNs)
learn millions of parameters associated with a series of transformations
defined by the model architecture, resulting in high-dimensional,
difficult-to-interpret internal representations of input data. As DNNs become
more ubiquitous across multiple sectors of our society, there is increasing
recognition that mathematical methods are needed to aid analysts, researchers,
and practitioners in understanding and interpreting how these models' internal
representations relate to the final classification. In this paper, we apply
cutting edge techniques from TDA with the goal of gaining insight into the
interpretability of convolutional neural networks used for image
classification. We use two common TDA approaches to explore several methods for
modeling hidden-layer activations as high-dimensional point clouds, and provide
experimental evidence that these point clouds capture valuable structural
information about the model's process. First, we demonstrate that a distance
metric based on persistent homology can be used to quantify meaningful
differences between layers, and we discuss these distances in the broader
context of existing representational similarity metrics for neural network
interpretability. Second, we show that a mapper graph can provide semantic
insight into how these models organize hierarchical class knowledge at each
layer. These observations demonstrate that TDA is a useful tool to help deep
learning practitioners unlock the hidden structures of their models.
