  Cancer of unknown primary (CUP) is an enigmatic group of diagnoses where the
primary anatomical site of tumor origin cannot be determined. This poses a
significant challenge since modern therapeutics such as chemotherapy regimen
and immune checkpoint inhibitors are specific to the primary tumor. Recent work
has focused on using genomics and transcriptomics for identification of tumor
origins. However, genomic testing is not conducted for every patient and lacks
clinical penetration in low resource settings. Herein, to overcome these
challenges, we present a deep learning-based computational pathology
algorithm-TOAD-that can provide a differential diagnosis for CUP using
routinely acquired histology slides. We used 17,486 gigapixel whole slide
images with known primaries spread over 18 common origins to train a multi-task
deep model to simultaneously identify the tumor as primary or metastatic and
predict its site of origin. We tested our model on an internal test set of
4,932 cases with known primaries and achieved a top-1 accuracy of 0.84, a top-3
accuracy of 0.94 while on our external test set of 662 cases from 202 different
hospitals, it achieved a top-1 and top-3 accuracy of 0.79 and 0.93
respectively. We further curated a dataset of 717 CUP cases from 151 different
medical centers and identified a subset of 290 cases for which a differential
diagnosis was assigned. Our model predictions resulted in concordance for 50%
of cases (\k{appa}=0.4 when adjusted for agreement by chance) and a top-3
agreement of 75%. Our proposed method can be used as an assistive tool to
assign differential diagnosis to complicated metastatic and CUP cases and could
be used in conjunction with or in lieu of immunohistochemical analysis and
extensive diagnostic work-ups to reduce the occurrence of CUP.
