  Objective: To introduce a method for tracking results and utilization of
Artificial Intelligence (tru-AI) in radiology. By tracking both large-scale
utilization and AI results data, the tru-AI approach is designed to calculate
surrogates for measuring important disease-related observational quantities
over time, such as the prevalence of intracranial hemorrhage during the
COVID-19 pandemic outbreak. Methods: To quantitatively investigate the clinical
applicability of the tru-AI approach, we analyzed service requests for
automatically identifying intracranial hemorrhage (ICH) on head CT using a
commercial AI solution. This software is typically used for AI-based
prioritization of radiologists' reading lists for reducing turnaround times in
patients with emergent clinical findings, such as ICH or pulmonary embolism.We
analyzed data of N=9,421 emergency-setting non-contrast head CT studies at a
major US healthcare system acquired from November 1, 2019 through June 2, 2020,
and compared two observation periods, namely (i) a pre-pandemic epoch from
November 1, 2019 through February 29, 2020, and (ii) a period during the
COVID-19 pandemic outbreak, April 1-30, 2020. Results: Although daily CT scan
counts were significantly lower during (40.1 +/- 7.9) than before (44.4 +/-
7.6) the COVID-19 outbreak, we found that ICH was more likely to be observed by
AI during than before the COVID-19 outbreak (p<0.05), with approximately one
daily ICH+ case more than statistically expected. Conclusion: Our results
suggest that, by tracking both large-scale utilization and AI results data in
radiology, the tru-AI approach can contribute clinical value as a versatile
exploratory tool, aiming at a better understanding of pandemic-related effects
on healthcare.
