  Profiling data by plotting distributions and analyzing summary statistics is
a critical step throughout data analysis. Currently, this process is manual and
tedious since analysts must write extra code to examine their data after every
transformation. This inefficiency may lead to data scientists profiling their
data infrequently, rather than after each transformation, making it easy for
them to miss important errors or insights. We propose continuous data profiling
as a process that allows analysts to immediately see interactive visual
summaries of their data throughout their data analysis to facilitate fast and
thorough analysis. Our system, AutoProfiler, presents three ways to support
continuous data profiling: it automatically displays data distributions and
summary statistics to facilitate data comprehension; it is live, so
visualizations are always accessible and update automatically as the data
updates; it supports follow up analysis and documentation by authoring code for
the user in the notebook. In a user study with 16 participants, we evaluate two
versions of our system that integrate different levels of automation: both
automatically show data profiles and facilitate code authoring, however, one
version updates reactively and the other updates only on demand. We find that
both tools facilitate insight discovery with 91% of user-generated insights
originating from the tools rather than manual profiling code written by users.
Participants found live updates intuitive and felt it helped them verify their
transformations while those with on-demand profiles liked the ability to look
at past visualizations. We also present a longitudinal case study on how
AutoProfiler helped domain scientists find serendipitous insights about their
data through automatic, live data profiles. Our results have implications for
the design of future tools that offer automated data analysis support.
