  Ductal carcinoma in situ (DCIS) is a non-invasive breast cancer that can
progress into invasive ductal carcinoma (IDC). Studies suggest DCIS is often
overtreated since a considerable part of DCIS lesions may never progress into
IDC. Lower grade lesions have a lower progression speed and risk, possibly
allowing treatment de-escalation. However, studies show significant
inter-observer variation in DCIS grading. Automated image analysis may provide
an objective solution to address high subjectivity of DCIS grading by
pathologists.
  In this study, we developed a deep learning-based DCIS grading system. It was
developed using the consensus DCIS grade of three expert observers on a dataset
of 1186 DCIS lesions from 59 patients. The inter-observer agreement, measured
by quadratic weighted Cohen's kappa, was used to evaluate the system and
compare its performance to that of expert observers. We present an analysis of
the lesion-level and patient-level inter-observer agreement on an independent
test set of 1001 lesions from 50 patients.
  The deep learning system (dl) achieved on average slightly higher
inter-observer agreement to the observers (o1, o2 and o3)
($\kappa_{o1,dl}=0.81, \kappa_{o2,dl}=0.53, \kappa_{o3,dl}=0.40$) than the
observers amongst each other ($\kappa_{o1,o2}=0.58, \kappa_{o1,o3}=0.50,
\kappa_{o2,o3}=0.42$) at the lesion-level. At the patient-level, the deep
learning system achieved similar agreement to the observers
($\kappa_{o1,dl}=0.77, \kappa_{o2,dl}=0.75, \kappa_{o3,dl}=0.70$) as the
observers amongst each other ($\kappa_{o1,o2}=0.77, \kappa_{o1,o3}=0.75,
\kappa_{o2,o3}=0.72$).
  In conclusion, we developed a deep learning-based DCIS grading system that
achieved a performance similar to expert observers. We believe this is the
first automated system that could assist pathologists by providing robust and
reproducible second opinions on DCIS grade.
