  Purpose: To evaluate a fully-automated deep-learning-based method for
assessment of intracranial carotid artery calcification (ICAC). Methods: Two
observers manually delineated ICAC in non-contrast CT scans of 2,319
participants (mean age 69 (SD 7) years; 1154 women) of the Rotterdam Study,
prospectively collected between 2003 and 2006. These data were used to
retrospectively develop and validate a deep-learning-based method for automated
ICAC delineation and volume measurement. To evaluate the method, we compared
manual and automatic assessment (computed using ten-fold cross-validation) with
respect to 1) the agreement with an independent observer's assessment
(available in a random subset of 47 scans); 2) the accuracy in delineating ICAC
as judged via blinded visual comparison by an expert; 3) the association with
first stroke incidence from the scan date until 2012. All method performance
metrics were computed using 10-fold cross-validation. Results: The automated
delineation of ICAC reached sensitivity of 83.8% and positive predictive value
(PPV) of 88%. The intraclass correlation between automatic and manual ICAC
volume measures was 0.98 (95% CI: 0.97, 0.98; computed in the entire dataset).
Measured between the assessments of independent observers, sensitivity was
73.9%, PPV was 89.5%, and intraclass correlation was 0.91 (95% CI: 0.84, 0.95;
computed in the 47-scan subset). In the blinded visual comparisons, automatic
delineations were more accurate than manual ones (p-value = 0.01). The
association of ICAC volume with incident stroke was similarly strong for both
automated (hazard ratio, 1.38 (95% CI: 1.12, 1.75) and manually measured
volumes (hazard ratio, 1.48 (95% CI: 1.20, 1.87)). Conclusions: The developed
model was capable of automated segmentation and volume quantification of ICAC
with accuracy comparable to human experts.
